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PREFACE 


These  proceedings  of  the  Second  Symposium  on  Long  Range  Forecasting  and 
Planning  contain  the  papers  which  were  presented  and  the  discussion  of  those  papers. 
The  discussion  was  recorded  during  the  symposium,  and  has  been  edited  to  clarify  the 
discussants’  meaning.  However,  during  the  editing  process,  a  conscious  attempt  was 
made  to  retain  the  flavor  of  free  discussion  which  me’’  «d  tne  symposium.  We  feel  that 
the  interchange  of  information  and  opinion  during  tne  discussion  adds  significantly  to 
the  formal  presentations,  and  we  hope  that  both  the  papers  and  the  discussion  will  prove 
to  be  of  value  to  all  those  concerned  with  methods  of  long-range  forecasting  and 
planning.  The  reader  should  remember  that  the  opinions  presented  by  the  ::peaker8, 
particularly  during  the  discussion,  are  their  own,  and  do  not  necessarily  represent  the 
official  position  of  their  organizations. 

Special  thanks  are  due  to  Mrs.  F.  E.  Carter,  from  the  Office  of  the  Staff  Judge 
Advocate,  Holloman  AFB,  for  recording  the  discussion  of  the  formal  papers;  and  to 
Colonel  Weldon,  the  Staff  Judge  Advocate,  for  making  Mrs.  Carter  available  for  this 
task.  Also  we  wish  to  acknowledge  the  valuable  assistance  of  Majors  Claude  Stephenson 
and  Eugene  Erb  in  editing  the  transcript  of  the  discussions. 

We  wish  to  acknowledge  the  efforts  of  Mrs.  Ann  C.  Masi,  Mrs.  Wanda  Climenhaga, 
and  Mrs.  Patricia  Mergler  in  preparing  the  manuscript  for  publication. 
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The  Delphi  procedure  was  developed  a!:  a  tool  for  helping  a  panel  of  experts  reach 
a  consensus.  While  this  consensus  may  be  about  any  subject  at  all,  one  of  the  interesting 
applications  of  the  Delphi  procedure  is  producing  forecasts.  Dr,  Dalkey  describes  the 
procedure,  with  some  information  about  its  histoiy  and  current  usage. 


DELPHI 


Norman  C.  Dolkoy,  Rand  Corp. 


INTRODUCTION 


Delphi  is  the  name  of  a  set  of  procedures  for  eliciting 
and  refining  the  opinions  of  a  group  of  people,  in  prac¬ 
tice,  the  procedures  would  be  used  with  a  group  of 
experts  or  especially  knowledgeable  individuals. 

The  significance  of  the  Delphi  technique  should  be 
examined  in  the  context  of  what  I  call  the  advice  com¬ 
munity.  Both  industry  and  government  are  served  by  a 
large  group  of  consultants  who  purvey  information, 
predictions,  and  analyses  to  aid  the  formation  of  policy 
and  making  decisions.  The  community  is  a  highly  mis¬ 
cellaneous  assortment  of  in-house  advisors,  and  exter¬ 
nal  consultants  from  academia,  other  industries,  non¬ 
profit  corporations,  and,  of  course,  any  other  walk  of 
life  that  appears  relevant  to  the  problem  facing  the 
decisionmaker.  Some  of  this  advice  is  based  on  solid 
generalizations  from  observation,  either  of  the  crude 
empirical  variety  of  somewhat  more  prestigious  deduc¬ 
tions  from  established  scientific  principles.  A  great  deal 
of  it  is  opinion. 

The  notion  of  opinion  is  extremely  fuzzy,  but  with 
your  indulgence,  I  would  prefer  not  to  try  to  make  it 
piecise.  With  respect  to  the  interests  of  this  conference, 
I  believe  you  will  agree  that  in  the  area  of  long  range 
forecasting  of  technological  and  social  developments 
there  is  an  especially  large  admixture  of  opinion.  For 
this  area,  the  creation  of  techniques  for  refining  opin¬ 
ion  is  of  particular  interest. 

Pragmatically,  a  basic  characteristic  of  opinion  as 
opposed  to  more  solid  knowledge  is  the  fact  that  if  you 
interrogate  several  equally  competent  individuals,  you 
are  likely  to  get  a  divergence  of  answers.  This  is  obvi¬ 
ously  not  a  defining  characteristic,  since  uniformity  of 


response  does  not  guarantee  the  solidity  of  that 
response.  From  the  standpoint  of  the  decisionmaker,  a 
divergence  of  estimate  creates  a  problem  of  how  to  use 
the  estimates  in  fashioning  his  policies.  There  are  sev¬ 
eral  heuristic  devices  that  are  traditional  in  the  advic'i 
community.  One  is  to  select  a  single  advisor  on  some 
grounds  (ranging  all  the  way  from  personal  friendship 
to  lustre  within  the  community).  This  usually  guaran¬ 
tees  a  certain  uniformity.  Another  is  to  involve  several 
knowledgeable  individuals  and  employ  some  method  of 
group  interaction  to  arrive  at  a  common  opinion.  The 
most  popular  of  such  methods  is  that  of  the  committee, 
or  commission,  with  a  variety  of  informal  ways  to 
arrive  at  the  sense  of  the  committee. 

Selection  of  a  single  advisor  in  soft  areas  is  clearly 
fraught  with  danger;  oi.  ‘he  other  hand,  committees 
have  certain  drawbacks  which  have  been  dramatized 
by  a  large  number  of  investigations  by  psychologists 
and  small-group  sociologists  over  the  past  two  decades 
(i).  One  major  drawback  is  the  influence  of  the  domi¬ 
nant  individual.  A  quite  convincing  series  of  studies 
have  shown  that  the  group  opinion  is  likely  to  be  highly 
influenced,  if  not  determined,  by  the  views  of  the  mem¬ 
ber  of  the  group  who  does  the  most  talking,  and  that 
there  is  no  significant  correlation  between  success  in 
influencing  the  group  and  competence  in  the  problem 
being  discussed.  Another  difficulty  which  has  not  re¬ 
ceived  as  much  attention  in  the  literature  is  noise — 
irrelevant  or  redundant  material  that  obscures  the  di¬ 
rectly  relevant  material  offered  by  participants.  A  third 
difficulty  is  the  group  pressure  that  overvalues  reaching 
a  consensus  (“A  committee  is  something  that  designs  a 
camel  when  you  want  a  horse.”) 


The  Delphi  procedures  have  been  designed  to  reduce 
the  effects  of  these  undesirable  aspects  of  group  inter¬ 
action.  The  procedure  has  three  distinctive  character¬ 
istics: 


DELPHI  PROCEDURES 

1.  Anonymity. 


2.  Controlled  feedback. 

3.  Statistical  group  response. 

Anonymity  is  a  cevice  to  reduce  the  effect  of  the 
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socially  dominant  individual.  It  is  maintained  by  elicit¬ 
ing  separate  and  private  answers  to  prepared  questions. 
Ordinarily,  the  procedure  is  carried  out  by  written 
questionnaire;  on-line  computers  have  been  used  for 
some  exercises.  All  other  interactions  between  respond¬ 
ents  is  through  formal  communication  channels  con¬ 
trolled  by  the  experimenters. 

Controlled  feedback  is  a  device  to  reduce  noise 
(among  other  things).  A  Delphi  exercise  wiii  usually 
consist  of  several  iterations  where  the  results  of  the 
previous  iteration  are  fed  back  to  the  respondents,  nor¬ 
mally  in  sununarized  form. 

As  a  representative  of  the  group  opinion,  some  form 
of  statistical  index  is  reported.  For  cases  where  the 
group  task  is  to  estimate  a  numerical  quantity,  the 
median  of  individual  estimates  has  turned  out  to  be  the 
most  useful  index  tried  to  date.  Thus,  there  is  no  par¬ 
ticular  attempt  to  arrive  at  unanimity  among  the  re¬ 
spondents,  and  a  spread  of  opinions  on  the  final  round 
is  the  normal  outcome.  This  is  a  further  device  to 
reduce  group  pressure  toward  conformity. 

A  typical  exercise  is  initiated  by  a  questionnaire 
which  requests  estimates  of  a  set  of  numerical  quanti¬ 
ties,  e.g.,  dates  at  which  technological  possibilities  will 
be  realized,  or  probabilities  of  realization  by  given 
dates,  levels  of  performance,  and  the  like.  The  results 
of  the  first  round  will  be  summarized,  e.g.,  as  the 
median  and  interquartile  range  of  the  responses,  and 
fed  back  with  a  request  to  revise  the  first  estimates 
where  appropriate.  On  succeeding  rounds,  these  indi¬ 
viduals  whose  answers  deviate  markedly  from  the 
median  (e.g.,  outside  the  interquartile  range)  are  re¬ 
quested  to  justify  their  estimates.  These  justifications 
are  summarized,  fed  back,  and  counterarguments  elic¬ 
ited.  The  counterarguments  arc  in  turn  fed  back  and 


additional  reappraisals  collected.  This  basic  pattern 
has,  of  course,  many  possible  variants,  only  a  few  of 
which  have  been  tried. 

The  procedure  has  been  exercised  with  material 
where  there  is  no  immediate  way  to  evaluate  the  re¬ 
sults — e.g.,  long-range  technological  and  social  develop¬ 
ments — and  also  with  material  where  there  is  the  possi¬ 
bility  of  checking,  such  as  short-range  economic 
predictions  and  estimates  of  quantities  where  the  actual 
figures  are  obtainable,  typically  almanac-type  material. 
For  material  where  confirmation  is  possible,  typical 
outcomes  are  that  opinions  tend  to  converge  during  the 
experiment,  and  more  frequently  than  not,  the  median 
response  moves  in  the  direction  of  the  true  answer.  In 
the  case  of  material  where  confirmation  is  not  possible, 
all  we  can  say  is  that  opinions  do  converge  during  the 
exercise  (2),  (3). 

One  additional  feature  of  present  Delphi  procedures 
should  be  mentioned.  Respondents  are  requested  to 
make  some  form  of  self-rating  with  respect  to  the 
questions.  Several  different  kinds  of  self-ratings  have 
been  tried — ranking  the  questions  in  the  order  of  the 
respondents  judgment  as  to  his  comjjetence  to  answer 
them;  furnishing  an  absolute  estime  of  the  respondent’s 
confidence  in  his  answer;  estimating  a  relative  self- 
confidence  with  respect  to  some  reference  group.  In 
general  there  has  been  no  significant  correlation  dis¬ 
covered  between  such  self-ratings  and  individual  per¬ 
formance  for  confirmable  estimates.  However,  it  has 
usually  been  possible  to  use  the  self-ratings  to  select  a 
subgroup  of  relatively  more  confident  individuals  where 
the  performance  of  the  subgroup  has  been  slightly,  but 
consistently  better  than  the  group  as  a  whole.  In  one 
very  thorough  study,  the  improvement  was  obtained 
only  by  combining  two  self-rating  indices — ranking  of 
questions,  and  absolute  estimates  of  confidence  {4) . 


RESULTS  OF  EXPERIMENTS 


There  an  many  things  we  do  not  understand  as  yet 
about  the  information  processing  going  on  during  a 
Delphi  exercise.  Thus,  we  cannot  as  yet  determine  how 
much  of  the  convergence  is  due  to  three  different  fac¬ 
tors  which  are  clearly  at  work:  (1)  Social  pressure, 
(2)  rethinking  the  problem,  (3)  transfer  of  informa¬ 
tion  during  feedback.  Several  exercises  have  been  con¬ 
ducted  that  throw  some  light  on  this.  In  one  (5) ,  a  set 
of  20  almanac-type  questions  were  posed  to  a  group  of 
23  respondents.  A  control  group  of  11  respondents  were 
given  the  same  questions,  but  on  the  second  round  were 
simply  asked  to  reassess  their  answers,  with  no  feed¬ 
back  whatsoever.  They  were  not  even  told  what  their 


previous  responses  had  been.  In  general,  except  for  two 
questions,  the  amount  of  convergence  was  comparable 
for  the  two  groups,  and  the  accuracy  of  responses  for 
the  control  group  was  as  good  as  for  the  second-round 
responses  of  the  experimental  group.  This  would  appear 
to  indicate  that  a  major  factor  in  this  exercise  was  re¬ 
thinking.  However,  the  effect  of  social  pressure  and/or 
information  transfer  is  also  indicated  by  the  fact  that 
for  the  experimental  group  the  interquartile  ranges  of 
the  .second  round  responses  were  uniformly  contained 
in  the  interquartile  ranges  of  the  initial  responses, 
whereas  for  the  control  group  the  second-round  ranges 
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were  contained  in  the  initial  ranges  for  only  13  out  of 
the  20  questions. 

To  try  to  pin  down  a  little  more  the  factors  involved, 
we  conducted  aii  experiment  this  summer  comparing 
the  performance  of  structured  face-to-face  discussion 
groups  and  the  anonymous  questionnaire  technique. 
The  experiment  was  guided  by  two  presumptions. 
(Hypotheses  is  too  pretentious  a  notion  in  this  rather 
unstructured  subject.)  The  first  presumption  was  that 
in  a  face-to-face  situation,  information  transfer  is  likely 
to  be  much  greater  than  in  the  anonymous  controlled 
communication  situation.  This  would  presumably  tend 
toward  greater  accuracy  on  the  part  of  the  conference 
estimates.  The  second  presumption  was  thi.t  the  effect 
of  undesirable  social  interactions  could  be  ameliorated 
by  imposing  a  specific  format  for  the  discussions.  The 
format  employed  was:  for  each  question  a  new  discus¬ 
sion  leader  was  selected  by  chance;  the  leader  listed  on 
a  blackboard  all  relevant  information  (including  opin¬ 
ions)  suggested  by  members  of  the  group;  he  then 
listed  as  many  different  approaches  (little  models)  for 
answering  the  question  as  the  group  could  devise;  esti¬ 
mates  were  made  by  each  approach ;  and  finally  a  group 
consensus  was  arrived  at  by  informal  agreement. 

The  presumption  to  be  tested  was  that  a  structured 
conference  of  this  sort  would  produce  more  accurate 
estimates  than  the  questionnaire  technique.  The  experi¬ 
ment  was  performed  using  a  group  of  graduate  students 
engaged  in  summer  consultant  activities  at  RAND. 
There  were  10  participants,  divided  into  two  groups  of 
five.  There  were  20  questions,  of  the  almanac  sort, 
divided  into  four  groups  of  five.  Each  participant  group 
answered  10  of  the  questions  by  questionnaire,  and  10 
by  structured  discussion.  The  only  innovation  in  the 
Delphi  procedure  was  to  interpose  a  pure  information 
round  between  the  first  and  second  estimation  round. 
Each  respondent  was  allowed  to  ask  the  group  two 


questions  and  the  group  replies  were  fed  back  before 
the  second  estimate  was  made. 

The  major  outcome  of  the  experiment  was  that  the 
presumption  that  the  structured  discussion  would  turn 
in  a  better  performance  was  not  borne  out ;  in  fact,  the 
questionnaire  responses  were,  if  anything,  somewhat 
more  accurate  than  the  structured  conference  responses. 
The  difference  was  not  significant  except  for  one  meas¬ 
ure,  namely  the  sums  of  ranks  of  standard  scores,^  in 
which  the  questionnaire  technique  showed  up  as  better. 

For  the  discussion  groups,  no  adequate  measure  was 
obtained  for  the  role  of  dominant  members,  noise,  and 
pressure  for  consensus;  but  it ’  's  rlear  from  observa¬ 
tion  of  the  discussions  that  the  at.uclure  imposed  was 
inadequate  to  eliminate  these  effecc. 

An  interesting  anomaly  appeared  in  the  performance 
of  the  questionnaire  groups;  namely,  the  responses  on 
the  second  round  were  more  accurate  than  the  responses 
on  the  fourth  (and  final)  round.  Whether  this  was  due 
to  fatigue — for  each  set  of  five  questions,  the  entire  set 
of  responses  was  obtained  in  one  afternoon  session — 
or  due  to  a  saturation  effect  (all  of  the  relevant  infor¬ 
mation  elicited  by  the  second  round,  and  simply  wan¬ 
dering  estimates  from  then  on)  cannot  be  determined 
from  the  data. 

Perhaps  the  significance  of  the  experiment  can  be 
most  sharply  summed  up  by  the  following  conclusion: 
If  the  conference  groups  had  been  requested  to  open 
their  session  with  anonymous  individual  guestimates 
of  the  answer  to  each  question,  the  median  of  these 
off-the-cuff  guesses  would  have  been  more  accurate  than 
the  group  consensus  obtained  after  a  more  or  less 
thorough  discussion  of  the  subject. 

'Standard  scores  were  computed  by  dividing  the  group  esti¬ 
mate  by  the  true  answer.  The  40  responses  were  ranked  in  order 
of  accuracy,  and  the  sums  of  these  ranks  taken  for  each  configu¬ 
ration  (group,  method,  question  set).  The  analysis  of  variance 
for  the  sums  of  ranks  indicated  a  difference  between  the  two 
methods  significant  at  the  .05  level. 


DISCUSSION 


Delphi  procedures  are  still  in  an  experimental  stage 
with  regard  to  applications  to  the  advice  process.  The 
evidence  is  mounting  that  systematic  processing  of 
expert  opinion  can  produce  significant  improvements 
both  in  accuracy  and  reliability  (using  the  notion  of 
reliability  to  refer  to  the  range  of  estimates).  However, 
the  role  of  Delphi  procedures  within  the  corpus  of  fore¬ 
casting  techniques — extrapolation,  simulation,  demand 
analysis,  gami:ig,  etc. — has  not  been  established.  In 
particular,  there  are  no  cases  that  I  know  of  where 


Delphi  procedures  have  been  explicitly  employed  to 
support  specific  policy  decisions.  Hence,  there  aie  no 
direct  comparisons  of  the  r  lative  effectivenesa  of  the 
procedures  versus  other  mote  traditional  forms  of 
advice.  Perhaps  Mr.  North  will  have  some  examples  to 
fill  this  void.  The  studies  that  I  am  familiar  with  in 
areas  relevant  to  policy  have  been  more  like  exploratory 
exercises  to  test  the  feasibility  and  manageability  of  tire 
procedures  with  extensive  subject  matters  and  geo¬ 
graphically  scattered  e.xperts.  In  this  respect,  the  pro- 
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cedures  have  turned  out  to  be  manageable,  but  often 
rather  cumbersome. 

A  common  reaction  is  to  imagine  Delphi  as  a  method 
of  obtaining  inputs  for  some  kind  of  formal  estimating 
structure — e.g.,  inputs  for  a  simulation  model.  I  must 
confess  that  at  times  I  find  this  is  an  appealing  notion, 
but  it  cannot  be  the  full  story.  Most  often,  for  those  areas 
where  data  is  lacking,  a  formal  model  is  lacking  as  well. 
As.  a  matter  of  fact,  the  Delphi  procedure  is  one  of  the 
most  efficient  ways  I  know  for  uncovering  the  implicit 
models  that  lie  behind  opinions  in  the  soft  areas.  One 
of  the  most  valuable  side  products  of  a  Delphi  exercise 
concerned  with  strategic  bombing  was  the  skeleton  of  a 
model  which  was  later  fleshed  out  in  great  detail  (2) . 

There  are  several  tautologies  which  are  directly  rele¬ 
vant  to  the  group  estimation  process;  (a)  The  total 
amount  of  information  available  to  a  group  is  at  least 
as  great  as  that  available  to  any  member,  (6)  the 
median  response  to  a  numerical  estimate  is  at  least  as 
good  as  that  of  one  half  of  the  respondents,  (c)  the 
amount  of  misinformation  available  to  the  group  is  at 
least  as  great  as  that  available  to  any  member  (this  one 
is  usually  overlooked  in  discussions  of  the  advantages 


(1)  Maies,  Nohman  R.  F.,  “Assets  and  Liabilities  in  Group 

Problem  Solving:  The  Need  for  an  Integrative  Function." 
Psychological  Review,  vol.  74,  No.  4,  July  1%7,  pp.  239- 
249. 

(2)  Dalkey,  N.,  and  0.  HELMca,  An  Experimental  Application 

of  the  Delphi  Method  to  the  Use  of  Experts.  “Manage¬ 
ment  Science,”  1963,  9,  458-467. 

(3)  Gordon,  T.  J.,  and  0.  Helmer,  “Report  on  a  Long-Range 


of  groups  versus  individuals) ,  (d)  the  number  of  ap¬ 
proaches  (or  informal  models)  for  arriving  at  an  esti¬ 
mate  is  at  least  as  great  for  the  group  as  for  any 
member,  (e)  corresponding  item  for  approaches  as 
(c)  for  information.  For  simplicity  I  have  included 
noise  in  misinformation  and  poor  approaches. 

These  tautologies  do  not  add  up  to  anything  like  a 
theory  of  the  group  estimation  process,  but  they  are 
suggestive.  For  example,  (c)  and  (e)  hint  that  there 
may  be  an  optimal  size  of  group  for  a  given  kind  of 
estimation.  This  would  be  in  accordance  with  some 
experimental  results  with  small  discussion  groups.  They 
also  suggest  that  part  of  the  group  estimation  process 
should  be  concerned  with  active  suppression  of  misin¬ 
formation  as  well  as  filling  voids  in  information. 

We  have  no  way  at  present  of  determining  whether 
the  questionnaire-feedback  procedure  is  anything  like 
an  optimal  use  of  the  information  available  to  a  group, 
or  whether  it  includes  a  mechanism  for  reducing  the 
effect  of  misinformation.  Nor  can  we  say  that  it  is  most 
effectively  used  in  isolation,  or  within  the  context  of 
other  methodologies.  In  short,  there  is  a  very  large  field 
waiting  for  the  plow. 


Forecasting  Study,”  the  Rand  Corp.,  p-2982,  September 
1964. 

(4)  Campbell,  Robert,  “A  Methodological  Study  of  the  Utili¬ 

zation  of  Experts  in  Business  Forecasting,”  uni'ublished 
Ph.  D.  dissertation,  UCLA,  1966. 

(5)  Brown,  B.,  and  0.  Hei.mei-  “Improving  the  Reliability  of 

Estimates  Obtained  from  a  Consensus  of  Experts,”  the 
Rand  Corp.,  p.  2986,  September  1964. 


Me.  Irw’N.  You  touched  rather  briefly  on  one  topic 
that  I  wondered  if  you  would  expand  on.  You  men¬ 
tioned  the  variance  to  the  questions,  in  your  twenty 
questions,  was  greater  than  the  variance  between  the 
control  groups.  Since  the  control  feedback  would  ob¬ 
viously  be  a  very  important  factor  in  Delphi  technique, 
the  person  who  controls  the  controls  and  his  attitude 
would  be  important.  Specifically,  I  was  wondering  in 
some  or  all  of  the  experiments  you  conducted,  do  the 
people  who  control  the  controls  know  the  answers? 

Dr.  Dalkey.  The  answer  is  they,  at  least,  have  the 
answer  available  and  probably  know  something  about 
it.  If  you  are  asking  the  question  whether  we  made  a 
•tudy  of  the  influence  of  that,  we  haven't.  I  would  like 
to  say  that  !s  an  important  factor  certainly  which  has 
not  been  looked  at.  You  are  quite  right  insofar  as  the 
feedback  is  controlled,  is,  in  fact,  manipulated  by  the 


experimenter.  The  experimenter  is  part  of  the  experi¬ 
mental  situation  in  this  case.  And  he  should  be  con¬ 
trolled.  He  has  not  been  controlled  in  the  experiments 
we  have  conducted. 

Mr.  Irwin.  It  seemed  to  me  particularly  important 
that  if  he  knows  the  answer,  he  is  going  to  filler  out 
some  noise  which  he  knows  is  noise  but  nobody  else 
does. 

Dr.  Dalkey.  Thtit's  correct. 

Dr.  Taylor.  We  were  just  visited  by  Richard  Suit- 
man  and  we  were  having  a  discussion  on  what  is  called 
rriterial  learning  versus  nonrriterial  learning.  This 
means  in  one  case  we  are  playi:  "  with  the  unknowns — 
no  one  knows  the  answer  yet — f  he  other  case  where 
the  experimenter  kni>ws  the  answer  and  these  may  be 
radically  different  areas  of  performance  and  different 
areas  of  experimentation.  Tliis  is  a  very  young  Held  in 
which  you  are  working,  as  you  indicate,  and  1  think  we 
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should  make  this  distinction  and  at  times  if  we  take 
truly  long  range  forecasting  and  planning,  I  think  we 
are  playing  with  this  noncriteria)  type  of  thing.  This  is 
just  an  observation. 

Dr.  Dalkey.  I  couldn’t  agree  with  you  more,  and  we 
have  made  some  attempts  to  distinguish  between  these 
two  types  of  material  in  the  studies  we  have  made. 
Unfortunately,  we  are  too  impatient  to  wait  20  years  to 
check  some  of  the  results.  It  is  clear  for  a  number  of 
very  important  issues,  you  simply  have  to  think  in  terms 
of  experiments  that  will  stretch  over  20  or  30  years, 
there  is  no  way  around  it. 

Colonel  Davis.  Perhaps  this  would  be  a  good  time  to 
mention  briefly  a  small  amount  of  work  we,  at  OAR, 
have  been  doing  in  the  Directorate  of  Plans  which 
seems  to  indicate  there  might  be  some  connection  be¬ 
tween  the  Delphi  technique,  for  example,  and  use  of  the 
growth  curve  or  logistic  function  in  order  to  describe 
the  evolution  of  a  technology  or  series  of  technologies. 
I  have  a  couple  of  viewgraphs  and  it  might  be  appro¬ 
priate  to  show  them  at  this  time.  First  of  all,  you  re- 


Figure  I-l 


a.  The  Cumulative  Logistic  Distribution 
(Hypothesis  for  Helmer  Forecast) 
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member  the  report  by  Gordon  and  Helmei  from  the 
Rand  Corp,  which  was  entitled  “Report  on  Long-Range 
Forecasting  Study.”  We  would  like  to  have  some  way 
of  getting  an  analytical  description  of  the  evolution  of 
technology  using  one  of  the  Delphi  group’s  predictions. 
In  this  case,  we  chose  the  predicted  progress  in  space, 
a  set  of  predictions.  In  the  report  there  was  a  graphical 
display  of  the  predicted  progress  in  space.  One  of  the 
axes  was  time  in  years  and  the  other  was  the  events 
which  wc  tool  to  be  arranged  in  some  degree  of  tech¬ 
nological  difficulty.  An  example  of  .some  of  the  events 
prcrlicted,  fitr  those  who  don’t  happen  to  remember 
them  in  detail  is:  Soviet  Union  orbital  and  U.S.  orbital 
rendezvous.  They  were  U>64  and  1%7  resjvectivelv. 
1970,  for  example,  U.S.  manned  lunar  flight  bv 
manned  scientific  orbital  .Uation,  10  .nen;  1975, 
something  like  ionic  propulsion,  man  to  Mars,  Venus 


flyby,  and  so  on.  In  trying,  at  least,  to  get  some  kind  of 
exponential  growth  related  to  this,  we  tried  to  plot  it 
and  found  that  a  plain  exponential  growth  didn’t  work 
but  a  cumulative  logistic  distribution  of  events  did,  in 
fact,  fit  within  some  reasonable  statistical  sense. 
First,  let  me  show  you  the  equation  (see  fig.  I-l). 
(1-G)  indicates  the  availability  of  knowledge  to 
be  exploited  which  is  occurring  in  a  random  fashion 
whereas  G  is  the  actual  growth  of  the  technology.  The 
abscissa  on  the  second  chart  (fig.  1-2)  is  made  up  of 
years,  0  up  to  40  and  is  keyed  to  the  year  1960  which  is 
the  year  0,  going  to  the  year  2000;  the  ordinate  r-epre- 
sents  a  measure  of  the  technological  difficulty  and  was 
obtained  from  Gordon  and  Helmer’s  results.  This  is 
simply  an  assumption  from  the  results  they  had  ob¬ 
tained  in  an  arbitrary  fashion  up  to  1984.  If  you  get 
the  arithmetic  mean  of  these  events  for  the  different 
years  and  plot  a  continuous  line  connecting  them,  you 
have  a  line  that  is  reminiscent  of  a  cumulative  distribu¬ 
tion  of  some  quantity.  Take  the  expression  just  shown 
and  make  the  appropriate  substitutions  then  you  get 
the  dashed  line  (fig.  1-2)  which,  when  you  apply  the 
chi  square  test  to  indicate  statistically  the  difference 
between  one  and  the  other,  is  not  unreasonable.  If  you 
wanted  to  reevaluate  when  the  events  might  have  oc¬ 
curred,  which  are  above  and  below  the  arithmetic  mean, 
you  might  shift  one  this  way  and  push  one  that  way  and 
get  a  reevaluation  of  the  actual  times.  In  conclusion, 
the  result  gives  an  indication  there  might  be  some 
connection  between  the  Delphi  technique  results  and  a 
previously  used  method  of  technological  forecasting. 
The  cumulative  logistic  distribution  has  been  used  to 
describe  the  growth  of  populations  and  growth  or 
development  of  other  events,  the  evolution  of  other 
quantities;  and  is  indicated  by  Dr.  de  Sola  Price  and 
other  writers  as  having  some  validity;  that  is,  a  cumu¬ 
lative  logistic  distribution  describes  the  evolution  even 
of  science  as  a  whole  and  presumably  this  is  associated 
technology. 

Dr.  Dalkey.  There  are  two  possible  hypotheses  we 
can  make,  one  is  that  this  is  an  estimate  on  the  part  of 
experts  as  to  the  way  in  which  technology  will  go  and 
the  other,  that  this  is  a  statement  about  the  psychology 
of  the  experts. 

rolone!  Davis.  Not  only  that  but  you  could  elucidate 
a  theory  of  growth  of  technology  as  described  bv  the 
product  of  a  couple  of  terma,  ( 1~G')  being  interprrtable 
as  the  cumulative  distribution  of  availability  of  knowl¬ 
edge  when  it  i»  being  exploited  against  time.  It  is  being 
expi(iu,d  oy  technologv.  In  other  words,  it  starts  out  at 
value  1  and  drops  on  down.  In  turn,  the  technologv 
gi>es  to  its  maximum  rap.shility  according  to  this  hvpo- 
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thesis  and  levels  off.  Also  1— C  is  recognizable  in  the 
distribution  of  physics.  The  distributing  of  quantum 
states,  some  type  of  order  situation,  where  only  one 
particular  level  can  be  occupied  by  one  particular 
quantum  state.  There  is  something  which  leads  you 
little  further  which  we  have  not  done  up  to  this  point  to 
see  if  there  is  further  correspondence  that  might  be 
interesting. 

Dr.  Johnston.  With  respect  to  this  logistic  curve,  we 
have  had,  before  my  time,  a  very  unfortunate  experi¬ 
ence  in  using  logistics  for  a  reason  that  may  be  relevant 
in  technological  control — that  is,  in  demography  you 
don’t  really  have  a  closed  space  except  in  some  ultimate 
sense  that  has  to  be  defined  out  of  the  physical,  perhaps 
the  psychological,  so  where  you  are  on  the  curve  today 
cannot  be  specified  nor  can  you  fix  the  indexes.  There 
may  be  an  ultimate  maximum  level.  In  the  field  of  tech- 
nology,  if  you  restricted  the  argument  to  a  particular 
area  of  development,  it  makes  sense  to  say  that  us  you 
approach  the  ultimate  which  you  agree  exists,  you  are 
going  to  have  a  harder  lime,  like  saying  to  get  1  percent 
down  is  a  lot  tougher  than  when  it  is  already  fairly 
high.  You  start  introducing  soap  and  you  get  good 
results.  Now  you  are  dealing  with  cancer  and  getting 
very  tiny  results.  I  don’t  know  if  you  are  talking  about 
technology  without  an  upper  limit  or  whether  it  is 
appropriate. 

Colonel  Davis  The  upper  limit  appears  to  be  pro¬ 
vided,  at  this  time,  by  the  estimates  of  the  Delphi  group 
involved  in  this  particular  exercise  and  some  of  the 
results  came  out  “never,”  which  puts  a  level  on  it;  how¬ 
ever,  the  upper  limit  is  Somewhat  in  doubt  and  will,  of 
course,  influence  the  whole  aljape  of  the  curve. 

Dr.  Linstone.  I  would  like  to  ask  a  question  of  Dr. 
Dalkey.  I  had  some  trouble  with  the  Delphi  method 
going  back  to  this  question  of  the  first  two  sociological 
statements.  Information  was  missing  which  gets  down 
to  the  problem  in  some  areas:  Is  the  majority  or  con¬ 
sensus  more  correct  than  the  maverick  opinion?  It 
seems  to  me  some  aretjs  can  apply — I’ll  just  mention 
two :  The  scientific  breakthroughs  beyond  the  year  2000 
on  the  scale;  another  area:  military  long-range  plan. 
Consider  as  an  example  the  military  area.  If  you  tried 
to  get  a  consensus  in  1956  on  army  mobility  in  1966, 
most  of  it  would  be  consensus  based  on  1956  which 
would  be  misleading  and  not  valid.  The  same,  for 
example,  in  an  historical  instance,  in  the  area  of  science 
when  the  gas  turbine  began  in  1940;  where  we  began, 
the  maverick  opinion  was  collecting  most  of  the 
records.  I  don’t  know  whether  Delphi  would  have 
helped;  when  you  get  such  problems  as  the  inbreeding 
of  ex{>erts  you  get  the  fact  that  breakthrough  is  un¬ 


predictable  by  definition  and  it  very  often  comes  in 
another  disciplinary  area.  The  estimates  are  in  one  area 
and  something  else  comes  from  the  outside.  I  wonder  if 
the  almanac  or  Lincoln  or  the  questions  you  mentioned 
are  valid  tests  for  these  kinds  of  applications  because  it 
is  one  thing  to  test  known  data,  obvious  to  the  expert; 
if  he  is  interrogated  properly  you  would  expect  a  group 
of  this  type  would  be  better  in. determining  it  than  an 
open  meeting.  By  putting  the  first  two  tautologies 
together,  what  do  you  gel?  I  would  like  a  comment. 

Dr.  Dalkey.  Yes,  I  am  hoping  Dr.  Taylor  will  raise 
and  discuss  more  thoroughly  the  area  of  evaluating 
experts.  We  haven’t  done  a  great  deal  of  work  on  this 
at  Rand  and  it  is  clearly  a  very  difficult  subject.  We 
haven’t  done  anything  about  this  problem  of  inbreed¬ 
ing  you  mention  which  I  think  is  very  serious.  There 
are  n  large  number  of  aspects  of  the  advice  community 
which  has  some  drawbacks  similar  to  those  of  smaller 
committees.  The  community  itself  pertains  to  some  of 
these  drawbacks  which  raises  difficulties — difficulties 
that  I  shall  not  comment  on  at  the  moment.  The  con¬ 
servatism  of  the  established  member  of  a  community  is 
very  undesirable.  With  regard  to  the  long-range  fore¬ 
casting  type  of  material,  obviously,  we  haven’t  been 
able  to  do  anything  experimental  in  that  area.  The 
question  whether  the  results  we  derive  from  experi¬ 
ments  with  confirmable  material  is  applicable  to  the 
nonconfirmable  material;  I  think  to  some  extent  it  is 
applicable.  That  is,  the  difference  between  postdiction 
and  prediction  is  not  as  great  as  we  sometimes  would 
like  to  think.  A  question  like  the  popular  vote  for 
Lincoln,  in  a  sense  that  was  trying  to  make  a  prediction 
backwards.  I  don’t  believe  that’s  enormously  different 
from  making  a  prediction,  but  there  are  differences. 
As  far  as  using  a  group  consensus,  the  use  of  a  sta¬ 
tistical  measure  like  the  median  rather  than  a  more 
direct  way  of  getting  group  agreement,  I  believe  the 
results  we  have  obtained  from  the  experiments  are 
directly  applicable  in  that  case.  I  think.no  matter  what 
material  you  are  dealing  with,  the  use  of  a  more  ob-  ' 
jegtive  technique  for  arriving  at  the  group  response  will 
increase  the  accuracy  of  what  you  get  from  the  group. 

I  feel  that  what  it  does  is  to  get  rid  of,  or  at  least 
mollify,  ameliorate,  the  willingness  of  members  of  a 
group  to  throw  away  information  they  know  in  order 
to  arrive  at  a  group  response  by  allowing  differences 
of  opinions  to  remain  and  simply  reporting,  Here’s  the 
range  of  opinions  that  we  got,  and  the  median  seems  to 
be  about  the  best  way  of  putting  it  together.  By  doing 
that,  we  allow  a  maximum  of  information  to  remain 
in  the  decision. 
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Conunatifler  Hamilton.  Several  of  us  are  a  little  un¬ 
familiar  with  the  Delphi  technique.  We  are  wondering 
about  the  background  and  development.  Do  you,  at 
Rand,  have  a  special  group  involved  in  this  technique 
or  is  this  a  sideline?  You  may  have  touched  on  this  at 
your  symposium  last  year. 

Dr.  Taylor.  Can  we  get  the  reports  from  last  year 
describing  the  Delphi  technique? 

Dr.  Dalkey.  In  my  paper  I  have  some  references 
with  which  you  can  obtain  some  background  on 
Delphi.  As  far  as  historical  questions,  there  are 
studies  on  the  subject  that  have  been  going  on  at  Rand 
since  the  beginning  of  Rand  in  a  rather  low  key.  By 
low  key,  I  mean,  rarely  have  there  been  more  than  two 
or  three  senior  people  involved.  It  has  been  inter¬ 
mittent.  We  started  in  1948 — 1949 — looking  just  at 
this  aspect  of  anonymity.  We  asked  whether  it  was 
possible  to  take  a  set  of  estimates  from  a  group  and 
improve  them  by  statistical  advice.  This  was  applied  to 
all  kinds  of  questions  such  as  forecasting  the  outcome 
of  horseracing  among  other  things.  We  found  that  it 
was  possible  to  use  statistical  averaging  techniques  to 
get  results  that  were  better  than  most  responses  of  the 
respondents.  What  do  I  mean  by  that?  This  tautology 
says  the  median  is  better  than  half  of  the  estimates. 
That’s  tautology.  You  can  always  be  sure  that’s  true; 
however,  in  most  of  the  experimental  results  you  will 
find  the  following;  This  is  a  set  of  estimates;  that  is, 
the  quantity  we  are  trying  to  estimate  and  you  receive 
some  type  of  distribution  in  answers,  the  true  answer 
will  be  somewhere  on  this  scale.  In  most  of  the  experi¬ 
ments  the  median  is  closer  to  the  true  answer  than  more 
than  half  of  the  respondents.  It  would  be  closer  to  the 
true  answer  in,  say,  three-quarters,  five-eights  of  the 
cases.  It’s  not  something  you  can  derive  from  logic  but 
it  is  something  that  turns  out,  to  come  from  experiment, 
and  we  have  found  that  we  were  just  looking  for  statis¬ 
tical  group  responses.  That  is  a  rather  crude  way  of 
going  at  the  problem  and  we  gradually  introduced  the 
feedback  way  of  refining  the  opinions  around  1953-54. 
There  was  a  little  hiatus  around  that  time.  The  reason 
for  the  hiatus  was,  we  were  exploiting  some  of  the 
models  I  mentioned  earlier  which  we  discovered  by  the 
Delphi  technique.  I  think  it  is  fair  to  say  that  this  is 
the  reason  for  an  increased  interest  in  Rand  and  else¬ 
where  in  this  technique  has  been  he  emergence  of 
a  greater  interest  in  long-range  forecasting.  The  reason 
for  that  is,  this  is  an  area  where  we  don’t  have  a  great 
deal  of  information.  We  don’t  have  a  large  number  of 
theories.  If  you  are  going  to  make  decisions  based  on 
long-range  forecasts  you  are  going  to  use  opinions  and, 
therefore,  a  way  of  refining  those  opinions  seems  to  be 


particularly  important.  I  don’t  know  whether  that 
answers  your  question. 

Mr.  Cetron.  There  is  a  book  which  you  may  find 
useful  by  Gordon  and  Hclmer,  “Social  Technology” 
published  by  Basic  Books  (1966),  that  would  answer 
your  question.  I  am  one  of  the  believers  that  hindsight 
is  better  than  foresight  by  a  danuied  sight  and  I  believe 
the  real  key  question  is  control.  As  you  mentioned,  you 
can  control  results  if  you  know  what  the  results  were 
when  you  check  your  information  from  past  per¬ 
formances.  If  you  know  something  happens  and  have 
the  information  you  can  also  control  the  future  pretty 
well.  One  of  our  admirals  said  he  can  make  these  fore¬ 
cast  self-fulfilling  by  controlling  the  expenditures  of 
the  money.  I  think  we  find  that  some  of  these  things 
are  feasible  but  may  never  be  picked  up.  Your  forecast¬ 
ing  is  strictly  what  is  possible,  and  what  will  happen 
depends  on  what  resources  are  put  in.  Is  there  any 
information  or  background  work  on  this  particular 
area? 

Dr.  Dalkey.  Nota  great  deal.  Let  me  make  this  com¬ 
ment  on  the  role  of  Delphi  with  regard  to  decision 
processing  itself.  Normally,  the  decision  process  is  laid 
out  like  this;  You  have  a  series  of  strategies,  policies, 
courses  of  action,  that  you  can  carry  out;  a  set  of 
possible  states  of  nature  over  the  future,  which  will 
influence  the  outcome  of  those  strategies  and  the 
general  decision  process  then  consists  in  finding  a 
figure  of  merit  for  strategy  versus  the  outcome.  One 
way  i»  to  find  some  estimate  of  the  likelihood  or 
probability  of  the  state  of  nature,  and  then  use  some 
expected  value  as  a  way  of  scoring  the  strategies  and 
deciding  on  them.  There  are  a  couple  of  features  we 
have  not  gotten  and  have  not  appeared  a  great  deal  in 
the  literature  so  far,  but  are  obviously  on  the  periphery 
and  very  important.  First  of  all,  the  kinds  of  discount 
rates  you  are  going  to  use  in  scoring  policies  for  events 
that  are  far  in  the  future.  If  you  look  at  the  normal 
social  process,  you  would  swear  that  for  many  social 
developments,  it  is  enormously  high.  For  example,  the 
standard  one  I  use  for  this  purpose  is;  Probably  one 
of  the  elements  of  the  long-rapge  picture  that  is  most 
sharpiy  etched  in  the  minds  of  long-range  forecasters 
is  the  population  explosion.  It  is  also  one  of  the  things 
that  in  some  sense  has  the  greatest  social  value  of  any 
of  the  kinds  of  events  that  are  talked  about.  It  is  a 
disaster  of  the  first  order  that  we  are  looking  forward 
to  with  a  great  deal  of  certainty.  The  question  is  how 
niuch  is  being  done  about  it  at  the  present  time.  If  you 
want  to  look  at  this  question  as  a  social  discount  rate, 
how  much ‘of  our  resources  are  we  actually  spending, 
at  the  present  time,  with  regard  to  that  problem  and 
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the  answer  is  practically  none.  Most  of  the  investment 
we  are  making  at  the  present  time  with  regard  to  that 
problem  has  to  do  with  birth  control  and  you  can  see 
it  is  possible  to  discount  most  of  what  is  being  done 
regarding  birth  control  in  terms  of  present  demand.  If 
women  in  the  United  States  weren’t  interested  in  birth 
control  devices,  we  probably  wouldn’t  be  doing  any¬ 
thing  about  it.  If  we  didn’t  think  the  population  of 
India  was  already  too  big,  we  probably  wouldn’t  be 
doing  anything  about  it.  In  short,  here  is  this  great 
looming  disaster  facing  us  with  a  very  high  probability 
and  we  are  investing  practically  nothing.  In  that  case, 
the  social  discount  rate  is  enormous.  That’s  the  kind 
of  problem  where  we  can  use  Delphi  techniques  and 
I  don’t  know  any  other  technique  that  would  give  us  a 
handle.  I  don’t  know  whether  that  answers  your 
question. 

Dr.  Chacko.  You  raised  the  question  about  the 
■relationship  between  Delphi  techniques  with  other 
I  means  of  forecasting  and  as  I  listened  to  you,  I  thought 

[  there  was  one  large  area  wliere  Delphi  could  be  used 

I  and  has  not  been,  at  least  you  didn’t  tell  us,  namely, 

I  your  examples  of  almanac  situtations,  such  as  Lincoln’s 

j  popular  vote  in  1860.  I  believe  you  are  probably  throw- 

j  ing  away  a  considerable  amount  of  information  which 

'  relates  not  to  the  outcome  or  the  ultimate  results,  but 

I  to  the  logical  process  which  this  group  has.  The  role  of 

the  maverick  is  denied  when  you  use  the  median  of 

the  numbers.  Have  you  worked  on  the  logical  process 

of  the  participants  in  your  experiments? 

Dr.  Dalkey.  The  answer  is  no.  We  think  it  is  im¬ 
portant  but  we  haven’t  dug  into  it  in  an  experimental 
I  fashion. 

i 

i  Dr.  Chacko.  I  have  done  a  limited  experiment  in 

I  this  connection.  In  your  speech  you  mentioned  the 

1  application  of  Delphi  to  business  and  goverriment, 

I  however,  I  will  leave  the  business  part  to  Dr.  North, 

j  I  will  report  on  what  I  attempted  with  respect  to  the 

1  basis  of  forecasting  for  national  policy  making.  The 

J  features  I  wanted  were:  (1)  Situations  should  refer  to 

j  the  highest  policy  matters;  (2)  the  process  underlying 

the  policy  recommendation  should  be  highlighted;  and 
(3)  the  maverick  responses  should  be  highlighted  and 
not  ignored. 

What  I  developed  is  a  blufl-thrcat  matrix;  Every 
event  in  the  cold  war  was  identified  in  terms  of  one 
I  or  more  of  110  bluffs.  The  bluffs  are  simply  any 
I  characterizations  of  situations.  When  someone  says, 

I  “There  is  a  bomb  in  this  building,”  it  is  a  bluff  irre- 

i  spective  of  the  truth  of  the  matter.  It  becomes  a  threat 
the  instant  you  look  for  an  exit  even  though  you  may 
break  your  leg  trying  to  escape. 


What  I  did  was  identify  110  categories  which  all 
conceivable  events  of  control  could  again  be  cate¬ 
gorized.  Please  feel  free  to  add  to  the  110  bluffs. 
Corresponding  to  the  characterizations,  I  identified  90 
responses,  and  I  am  glad  it  came  out  as  a  110  times 
90  matrix  rather  than  100  times  100. 

I  presented  these  characterizations  to  the  team  at 
Bendix,  at  the  University  of  Michigan,  arms  control 
group.  It  was  a  group  of  scientists  whose  reaction  was 
quite  favorable.  Two  weeks  later,  I  gave  it  to  the  Joint 
War  Games  Agency  people.  There  we  had,  of  course, 
representatives  from  all  agencies  and  after  explaining 
the  rules,  I  had  them  play  the  Vietnam  game  using  only 
unclassified  information  as  the  bluff:  how  do  you  see 
the  war  situation  and  what  are  the  objectives.  The 
amazing  thing  was  that  in  spite  of  the  most  outlandish 
hypotheses  such  as  Cuba  having  a  hand  in  Vietnam, 
and  the  resolutions  they  wanted  to  make,  I  was  able  to 
show  that  each  man  had  a  large  matrix.  When  it  was 
put  together,  there  was  a  reasonable  area  of  consensus. 
First,  if  there  was  anyone  who  suggested  tactical  nu¬ 
clear  weapons,  that  would  show  up  as  a  different  “X” 
and  would  not  be  discarded.  Secondly,  we  are  forcing 
each  man  to  say  why  he  indicated  such  action,  which  I 
think  you  are  losing  if  you  merely  look  at  the  quantity 
results.  So  he  says,  “because  of  these,  therefore  I  say 
such  and  such.” 

The  merit  of  this  approach  is  that  instead  of  15 
volumes  of  500  or  600  pages,  the  President  of  the 
United  States  can  look  at  one  matrix  representing  not 
only  the  policy  recommendations  but  also  the  logic 
leading  to  the  recommendation. 

With  the  500-  or  600-page  volume,  the  President  is 
damned  if  he  doesn’t  follow  the  advice  on  the  last 
page  but  he  is  also  damned  if  he  does,  because  the 
499  pages  preceding  contain  caveats.  But,  with  the 
bluff-threat  matrix,  both  the  reasoning  and  the  recom¬ 
mendation  are  shown  and  any  maverick  is  not  ignored 
but  recognized.  The  orientation  is  toward  substantial 
policy  matters  of  national  concern. 

Major  Martino.  I’d  like  to  say  something  regarding 
the  control  exerted  by  the  experimenter.  From  the 
standpoint  of  the  use  of  the  ’Delphi  procedure  as  an 
advice  giving  or  advice  gaining  instrument  for  the 
)  olicy maker,  his  role  is  different  from  that  of  an 
experimenter  in  an  experiment  determining  the  be¬ 
havior  of  people  in  the  Delphi  situation.  If  he  wants 
to  alter  the  feedback  or  control  the  information  in,  a 
way  which  gives  predetermined  results,  if  he  wants  to 
kid  himself,  he  doesn’t  need  to  go  through  the  elabora¬ 
tion  of  the  Delphi  procedure.  The  real  danger,  as  I 
see  it,  is  a  man  may  be  kidding  himself  unconsciously. 
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When  I  ran  a  Delphi  experiment,  I  made  a  deliberate 
decision  at  the  outset  not  to  include  any  responses  of 
my  own  or  to  sneak  In  any  predictions  of  my  own 
because  I  believed  I  ran  the  danger  of  kidding  myself 
consciously  or  unconsciously.  In  short,  I  hired  a  team 
of  experts  and  if  I  had  been  an  expert  I  wouldn’t  have 
hired  them.  So  I  stayed  out  of  it.  But  I  would  like  to 
inquire  of  Dr.  Dalkey  if  any  other  attempts  have  been 
made,  or  if  you  have  looked  to  the  problem  of  un¬ 
consciously  fooling  yourself  when  you  are  the  policy¬ 
maker  who  has  hired  experts  to  run  through  a  Delphi 
sequence  and  give  you  advice. 

Dr.  Dalkey.  Let  me  first  distinguish  between  experi¬ 
ments  and  technical  applications  of  the  procedure. 
There  is  a  big  difference  between  using  the  Delphi 
procedure  as  a  tool  of  investigation  for  methodology 
and  using  it  as  a  tool  for  putting  information  in  a  form 
which  is  useful  to  a  decisionmaker.  These  arc  two  dif¬ 
ferent  types  of  operation.  One  is  an  application  of  the 
techniques  and  the  other  is  a  study  of  the  techniques 
and  the  information  processing  situation.  In  the  past 
this  distinction  hasn’t  been  as  sharp  as  we  would  like  it 
to  have  been,  but  a  number  of  exercises  which  have 
been  run  using  the  Delphi  techniques  have  really  not 
been  experiments.  They  have  been  tipplications  of  the 
procedure.  As  far  as  I  am  concerned,  when  you  arc  ap¬ 
plying  the  techniques  ther<i  are  no  holds  barred.  If  the 
experimenter  wants  tj  think  of  himself  as  being  part 
of  the  expert  community,  it  seems  to  me  that  is  a 
perfectly  reasonable  thing  to  do.  In  some  of  the  exer¬ 
cises  we  have  run,  of  the  applied  sort,  our  feeling  was 
that  after  the  exercise  was  finished,  the  group  had  in 
some  sense  a  kind  of  expertise  which  none  of  the  re¬ 
spondents  had.  They  had  a  better  grasp  of  the  entire 
field,  and  of  the  interrelationship  of  the  factors  than 
any  single  respondent.  It  is  a  way  of  helping  a  very 
small  group  of  experts  to  become  more  expert.  It  is  a 
way  for  them  to  become  acquainted  with  all  the  rich¬ 
ness  of  the  field;  to  become  as  clear  as  possible  as  to 
the  structure  of  that  field.  As  a  device  for  a  small  group 
of  experts  who  want  to,  in  a  sense,  become  as  rapidly 
acquainted  with  ail  the  richness  of  the  field  as  they  can,- 
it  seems  to  me  it  would  be  an  extremely  valuable  devide. 
That's  irrespective  of  the  other  qualities  of  the  tech¬ 
nique.  In  that  case,  it  is  not  only  reasonable  for  the 
experimenter  to  manipulate  the  material  before  he  feeds 
it  back,  it  is  reasonable  for  him  to  impose  some  struc¬ 
ture  on  the  feedback  material.  In  short,  if  you  think  in 
terms  of  several  rounds,  by  the  end  of  the  second 
round  of  the  experiment,  you  can  say,  “A  pattern  is 
emerging.  I  begin  to  see  what  this  field  looks  like.  Now 
let  me  check  my  intuitions  against  the  whole  panel  of 


experts.”  However,  as  I  say,  that  application  is  not  an 
experiment.  I  think  you  have  to  be  very  careful  in 
interpreting  the  results  of  that  kind  of  an  exercise.  In 
the  material  which  I  presented  this  morning,  I  tried  to 
be  careful  to  refer  to  exerci^  where  we  felt  the  amount 
of  interference  of  the  experimenter  had  been  sufficiently 
small  so  we  could  say  we  were  dealing  with  an  experi¬ 
ment  and  not  with  a  sure  exercise. 

Dr.  Oberbeck.  My  question  is  aimed  at  focusing 
attention  on  what  is  an  expert  and  what  do  you  mean 
by  the  accuracy  which  you  defined  with  respect  to  the 
almanac  event.  My  question  is:  In  what  respect  do  the 
Gallup  polls  meet  your  criteria  with  the  exception  of 
the  last  point  and  if  the  Gallup  poll  is  a  Delphi  tech¬ 
nique?  However,  if  we  know  exactly  how  the  Gallup 
polls  are  run,  then  what  does  expert  mean  there;  and 
what  does  accuracy  mean  there;  and  what  do  we  know 
about  accuracy  in  respect  to  the  Gallup  polls? 

Dr.  Dalkey.  I  am  not  an  expert  on  experts.  We 
haven’t  done  any  studies  which  really  illuminate  this 
enormously.  First  of  all,  presumably  an  expert  is  some¬ 
one  who  just  has  a  fair  amount  of  miscellaneous  back¬ 
ground  information  in  the  field;  and  if  you  take  the 
standard  type  of  definition  in  the  selection  of  experts, 
you  will  find  that’s  usually  the  kind  of  criteria  used. 
It  usually  comes  down  to  a  simpler  one  which  is  the 
number  of  publications  a  man  has  put  out.  It  is  a 
standard  academic  type  of  criterion,  and  that  is  a 
measure  of  the  amount  of  background  information 
people  have.  But  there  is  a  point  which  is  extremely  im¬ 
portant.  No  work  has  been  done,  but  there  is  some  in¬ 
formal  analyses  of  what  we  have  done  in  this  area  and 
there  is  a  big  distinction  between  having  a  wide  range 
of  background  information  and  being  able  to  use  that 
information  to  estimate  the  answers  to  new  questions. 
In  short,  there  seem  to  be  two  characteristics  involved 
in  what  you  would  like  to  think  of  as  an  expert:  A  lot 
of  background  information  and  then  some  kind  of  skill 
in  putting  that  information  together.  Let  me  give  the 
result  of  one  study  we  made  when  we  were  working 
with  the  statistical  response  itself  with  the  almanac 
type  of  question.  There  it  was  possible  to  ask  a  very 
large  number  of  questions  in  order  to  make  statistical 
analysis  of  the  results.  In  one  study,  we  asked  a  rather 
intricate  question.  The  kind  of  response  we  were  getting 
from  individuals  was  in  terms  of  the  probability  of  a 
statement  being  true.  These  were  statements  where  the 
answer  could  be  obtained.  We  deliberately  mixed  true 
and  false  statements  and  we  simply  asked  the  respond¬ 
ents  to  estimate  the  probability  that  the  statement  was 
true.  We  then  introduced  a  notion  of  precision,  the 
idea  of  precision  was  the  following:  Suppose  you  take 
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all  the  responses  of  a  given  individual  in  whicn  he  said 
the  likelihood  of  this  statement  being  true  is  75  percent. 
We  took  all  of  the  responses  in  groups.  For  e.'-.ample, 
take  75—80,  count  the  number  of  questions  where  he 
responded  75—80,  then  ask  what  was  the  proportion 
of  true  statements  in  that  group.  The  idea  was  that  a 
good  predictor  was  one  where,  roughly  speaking,  if 
you  said  75  -80,  then  75—80  percent  of  the  statements 
in  that  group  would  be  true.  We  discovered  that  almost 
all  of  the  respondents  were  good  in  this  respect.  If  you 
used  just  this  measure,  they  all  did  very  well.  This  was 


rather  startling;  however,  it  turned  out  that  there  were 
a  few  who  almost  never  used  50  or  60  percent.  They 
always  stayed  quite  well  up  in  the  75’s  80’s,  and  90’s. 
Others  were  very  fond  of  50,  60,  or  40.  There  was  one 
individual  who  was  very  fond  of  95  and  gave  a  very 
high  proportion.  Well,  95  and  5,  5-0,  95-100.  It  turned 
out  that  he  also  was  very  good  in  the  sense  that  when 
he  said  something  had  a  probability  of  95,  it  was  very 
likely  true.  If  he  said  it  had  a  probability  of  5,  it  was 
very  likely  false.  He  is  the  kind  of  chap  we  like  to  call 
an  expert.  We  didn’t  find  very  many. 


One  frequently  userl  metlu  d  of  generating  forecasts  is  to  ask  an  expert  in  the  field. 
This  leaves  another  problem,  however.  How  does  one  go  about  identifying  an  expert? 
Professor  Taylor  describes  some  of  his  research  on  means  for  identifying  various 
kinds  of  talents  in  individuals.  This  can  be  extremely  important,  as  his  work  has 
show  n  that  everyone  is  good  at  something,  and  no  one  is  good  at  everything.  An  expert 
in  the  field  to  be  forecast  may  ivell  turn  out  to  have  no  talent  at  all  for  foits  asting.  He 
would  thus  be  an  unfortunate  choice  as  u  s<,>urce  of  forecasts. 

—  Editor 
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UNSO'uVED  PROBLEMS  IN  SELECTING  EXPERTS  IN 
PLANNING  AND  LONG-RANGE  FORECASTING 

Calvin  W.  Taylor,  Univorsity  at  Utah 


It  has  been  both  an  opportunity  and  a  challenge  to 
prepare  a  presentation  on  this  topic.  Dr.  Dalkey  opened 
the  field  in  an  excellent  manner  with  his  clear  com¬ 
munications.  In  fact,  he  provoked  enough  thought  in 
me  for  a  second  entirely  different  speech  from  the  one 
I  will  give.  I  particularly  liked  his  analogy  that  the 
field  is  ready  for  the  plow,  with  which  I  strongly  agree. 

First  let  me  mention  briefly  some  other  fields  of 
talent  which  we  have  already  plowed  and  show  how 
these  enable  us  to  be  more  ready  now  to  plow  the 
fields  of  planning  and  forecasting.  My  approach  will 
be  a  more  broad  theoretical  one.  However,  the  many 
specifics  that  he  has  been  citing  would  provide  many 
research  opportunities  to  learn  more  about  those  who 
are  more  expert  from  those  less  expert.  His  last  com¬ 
ment  indicated  clearly  that  people  differ  quite  widely 
in  their  planning  and  forecasting  talents,  which  is  the 
heart  of  the  selection  problem.  Let  us  extrapolate  from 
what  we  know  in  some  other  high-level  talent  areas  and 
from  what  little  we  know  in  planning  and  forecasting 
talents  as  a  way  to  anticipate  what  likely  will  happen 
as  we  do  more  R.  &  D.  work  on  these  newer  talent  areas. 

First,  let  me  make  an  apology  for  our  field  for 
having  made  a  contribution  some  60  years  ago  and 
then  having  oversold  the  public,  or  perhaps  used  in¬ 
appropriate  words  to  have  misled  the  public.  If  we 
should  sketch  a  large  circle  to  represent  all  the  various 
things  that  this  tremendously  complex  brain,  or  man’s 
mind,  can  do,  we  put  together  some  6(  years  ago  a  test 
and  we  called  it  by  the  big  word  “in  diligence.”  Now 
that  we  have  hindsight,  which  is  far  better  than  our 
foresight,  we  can  see  that  intelligent  definitely  does 
not  measure  that  entire  large  circle  nut  measures  only 
one  small  circle  within  the  much  larger  circle.  With 
this  little  circle  being  intelligenc^  but  with  people 
thinking  we  are  talking  about  the/fcig  circle  instead  of 
the  little  circle,  then  you  realize /he  confusion  that  we 
have  created.  The  total  research  to  date  indicates  that 
something  like  two-thirds  of  t^at  large  circle  has  been 
explored.  We  have  found  at  least  six  or  seven  other 


circles  (or  talents)  in  the  areas  of  the  known,  which, 
on  the  average,  might  be  equal  in  size  and  importance 
to  that  first  talent  which  we  called  intelligence.  These 
other  talents  as  yet  have  been  largely  ignored  in  pro¬ 
grams  for  identifying  the  talented.  They  have  also  been 
neglected  in  educational  programs,  too,  because  we 
have  oversold  educators  on  the  word  “intelligence.” 

The  plan  for  the  intelligence  test  was  to  measure  what 
was  already  occurring  in  schools.  So  we  produced  a 
closed  system  of  intelligence  intercorrelated  with  grades 
and  grades  intercorrelated  with  intelligence — but  un¬ 
fortunately  grades  do  not  correlate  very  well  with 
what  one  does  in  the  work  world.  This  is  the  dilemma 
that  exists  and  our  profession  has  probably  been  as 
tardy  as  anyone  in  finding  out  that  this  is  a  closed 
system.  The  academic  world  is  one  world  and  the  world 
of  the  professions  and  other  high-level  .talents  is  an¬ 
other  world  and  there  is  comparatively  little  relation¬ 
ships  between  these  two  worlds.  In  one  case,  we  have 
800  correlations  between  performances  in  the  academic 
world  and  the  professional  world  and  the  average  of 
these  correlations  is  .00,  with  a  random  error  curve 
around  zero  describing  very  well  the  histogram  display 
of  all  these  correlations. 

We  have  also  looked  at  educational  practice  in  terms 
of  the  basic  research  on  human  characteristics.  We  took 
several  views  of  it,  one  of  which  states  that  in  this  third 
perspective,  education  is  viewed  in  relation  to  the  actual 
world-of-work  requirements.  Then  we  talk  about  the 
Ph.  D.  on-the-job  being  a  higher  degree  than  the  Ph.  D. 
in  the  academic  world  and  how  certain  effects  of  the 
academic  world  have  to  wear  off  before  a  person  can 
attain  the  real  degree  which  is  only  earned  by  becoming 
truly  productive  scientists.. 

My  doctoral  professor  was  L.  L.  Thurstonc,  a . 
mechanical  engineer  who  worked  for  Edison  before 
becoming  a  psychologist.  He  did  most  of  the  initial 
basic  work  on  what  were  called  primary  mental  abili¬ 
ties,  stretching  the  number  of  these  talents  or  abilities 
beyond  the  seven  or  so  in  intelligence  tests  to  about  20 
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before  his  death.  Guilford  and  his  students  have  since 
picked  this  up  and  developed  it  much  further.  If  we 
want  to  consider  Guilford’s  periodic  table  of  the  mind 
(The  Structure  of  Intellect,  see  Taylor,  1964),  approxi¬ 
mately  80  out  of  a  possible  120  cells  in  his  periodic 
table  can  be  identified  to  date.  The  intelligence  test 
includes  about  six  to  eight  or  if  we  are  very  generous, 
we  could  say  almost  10  out  of  the  80.  That  gives  you  a 
feeling  that  we  have  gone  a  long  ways  in  the  talent  area 
beyond  what  we  knew  60  years  ago,  except  in  com¬ 
municating  this  to  the  public  and  in  widening  the 
practices. 

One  of  these  other  new  talent  areas  on  which  we  have 
been  working  very  hard  is  creativity  and  another  one 
is  communication  abilities.  A  further  talent  area  which 
we  are  now  trying  to  open  up  concerns  planning  abili 
ties  or  talents.  Frankly,  we  have  not  yet  paid  much 
attention  to  the  equally  challenging  talent  area  of 
long-range  forecasting. 

Incidentally,  my  most  recent  experience  in  planning 
was  in  plantiing  this  speech.  At  times  I  couldn't  help 
but  wonder  if  I  had  a  good  enough  plan  and  pondered 
if  I  should  abandon  it  and  start  over.  As  indicated 
earlier,  I  was  sparked  to  a  new  se:  of  notes  from  the 
first  presentation  and  maybe  I  will  wish  afterward  that 
I  had  changed  the  plan  in  order  to  speak  from  these 
newest  and  freshest  notes. 

Let  me  tell  about  some  of  these  later  approaches  our 
profession  has  generally  used  in  these  newer  talent 
areas.  Creativity,  very  appropriately  by  its  nature, 
represents  our  breakaway  talent.  Through  our  findings 
on  creativity  we  are  trying  to  break  away  from  this 
8i;.)ngly  held  but  erroneous  notion  of  one  and  only 
one  kind  of  talent,  one  and  only  one  kind  of  giftedness. 
Contrary  :<>  this  widely  held  notion,  the  overwhelming 
evidence  is  for  multiple  types  of  talent  and  multiple 
types  of  giftedr  .-as. 

Let  me  open  up  the  work  on  creativity  with  my  story 
about  a  plan  for  geeth  g  rich  quickly,  though  obvioudy 
I  haven’t  put  the  plan  into  effect  yet.  1  will  tsli  you 
about  how  it  all  starts,  in  terms  of  life  and  creative 
talent  on  this  earth  for  a  given  indWidual.  I  hrve  a 
plan  to  open  a  nstional  chain  of  schools,  so  although 
I  am  here  officially  to  make  this  presentation,  on  the 
aide  I  am  laying  the  groundwork  to  open  another  one 
of  my  new  schools.  That  is  my  real  agenda  here.  Of 
course,  when  we  think  about  schools,  we  aak  at  what 
level  should  we  start  applying  our  creativity  work.  We 
always  come  up  with  the  answer  of  starting  at  preschool 
levels.  1  will  use  the  best  psychological  advertising.  All 
at  once  I  will  open  up  a  nationwide  chain  of  these 
nursery  schools  and  turn  on  flashing  noon  signs  across 


the  Nation.  The  signs  will  advertise:  “A  Nursery  School 
for  Creative  Children  Only.”  Then  someone  says,  “You 
still  have  a  problem  in  selecting  the  children.  How  are 
you  going  to  select  them?”  I  have  a  very  simple 
answer:  “I’m  just  going  to  let  the  parents  decide.” 
That’s  how  it  all  starts  as  far  as  creative  talent  in  one’s 
life  is  concerned. 

There  has  been  a  lot  of  research  on  what  happens  to 
potential  creative  talent  as  people  move  through  their 
academic  world  and  into  the  world  of  work.  Every  time 
we  review  these  findings,  we  always  want  to  go  back  as 
early  as  possible  to  start  at  preschool  ages  and  shoot 
the  works  to  implement  all  we  have  learned  about  cre¬ 
ativity  so  far.  Tomorrow  I  will  work  with  the  Colorado 
State  program  for  g’fted  and  creative  children.  Colo¬ 
rado  is  a  leader  in  being  aware  of  and  recognizing 
officially  at  least  two  kinds  of  giftedness.  We  are  going 
to  try  and  open  further  tomorrow  this  awareness  of 
their  4!!0  workers  throughout  their  State  to  include 
multiple  types  of  giftedness. 

Researchers  in  creativity  huve  been  asked  how  to 
start  having  our  findings  implemented  into  classrooms. 
One  way  I  responded  was  by  writing  a  series  on  some 
clues  to  creative  teaching,  on  some  hints  about  teaching 
for  creating  activities  in  the  students.  This  is  a  pattern 
I  would  like  to  see  followed  in  planning,  too,  by  writing 
a  series  and  by  developing  demonstrations  on  teaching 
for  planning  activities  in  the  students.  Through  these 
steps  eventually  tbs  students  can  have  the  opportunity 
to  experience  creative  processes  and  planning  processes 
and  can  develop  their  creative  talents  and  also  their 
planning  talents  during  their  regular  classroom 
programs. 

Recently  we  have  been  doing  many  biographical 
studies,  life  history  studies  of  creative  people,  especially 
scientists  and  engineers  ^as  published  Mar.  3,  1%7  in 
“Science”).  From  studying  their  life  programs,  i.e.. 
how  they  have  been  programed  to  date,  one  can  fore¬ 
cast  whether  they  are  likely  to  be  creative  or  productive 
or  not.  This  is  probably  ts  good  a  single  approach  as 
(vailable  for  forecasting  whether  a  person  will  be  pro¬ 
ductive  and  even  creatively  productive  in  science.  It 
is  also  our  best  single  approach  for  forecasting  perfor 
mances  beyond  the  academic  world  in  any  field.  Inside 
the  academic  world,  the  usual  study  has  been  to  predict 
school  grades.  Most  studies  have  done  that  and  most 
studies  aren’t  worth  very  mud.  because  grades  don’t 
forecajit  other  things  well,  except  other  academic 
grades.  This  is  one  of  the  real  dilemmas  in  the  field,  so 
I  am  pl^psed  to  work  with  you  to  start  breaking  further 
away  from  forecasting  performances  only  within  the 
academic  world. 
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Recently  we  have  finished  some  work  for  the  Peace 
Corps  in  which  we  were  trying  to  identify  those  with 
talents  and  characteristics  that  are  needed  overseas  in 
the  Peace  Corps,  new  talents  like  sensing  problems, 
resourcefulness,  initiating  action,  innovativeness,  and 
creativity.  We  have  been  studying  what  characteristics 
are  needed  to  be  top  performing  physicians  and  nurses, 
too,  so  we  are  not  studying  low  level  and  applied  fields 
after  one  leaves  a  so-called  high-level  academic  world, 
but  instead  we  are  focusing  upon  very  high  level  fields 
and  professional  performances  and  accomplishments. 

With  AFOSR  support  we  have  done  several  studies 
on  communication  abilities  and  creative  abilities  just 
published  as  a  GPO  monograph  (Taylor,  Ghiselin,  and 
Yagi,  1067).  In  these  studies  of  communication  abil¬ 
ities  needed  in  large  organizations  we  found  several 
high-level  talents,  such  as  revision  abilities  and  talking 
abilities,  to  be  needed  in  several  important  supervisory 


and  executive  activities;  we  also  found  that  many  of 
these  important  communication  talents  are  often  being 
ignored  in  usual  educational  programs.  Such  programs 
tend  to  emphasize  reading  end  writing  much  more  than 
revision  or  oral  communication,  including  talking  and 
listening  abilities.  Even  though  students  do  take  Eng¬ 
lish  and  language  arts  courses  from  12  to  16  years  in 
school,  we  have  evidence  that  the  kind  of  communica¬ 
tion  abilities  which  are  important  in  the  world  of  work 
are  not  ones  in  which  they  get  much  training.  Revision 
abilities  is  our  best  single  example  of  a  high-level 
talent  in  the  total  complex  set  of  communication  abili¬ 
ties  which  is  generally  neglected  in  educational 
programs. 

As  shown  in  figure  1,  our  theory  for  education  entails 
a  multiple-talent  approach.  We  are  learning  how  to 
develop  creative  and  other  new  talents  in  classrooms. 
We  have  strong  evidence  from  many  demonstration 
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itudies  that  two  things  are  happening  simultaneously 
in  the  students.  One  is  that  they  are  acquiring  subject 
matter  and  the  other  is  they  are  using  some  internal 
processes  to  deal  with  and  acquire  that  subject  matter. 
There  are  many  different  internal  processes  that  stu¬ 
dents  could  use  while  they  are  acquiring  subject  matter 
and  these  different  processes  could  be  and  are  right¬ 
fully  called  different  talents.  So  students  can  acquire 
subject  matter  by  the  academic  talents  (sometimes 
facetiously  called  the  tape  recorder  talents)  or  they  can 
acquire  subject  matter  by  creative  talents  or  they  can 
acquire  knowledge  by  decisionmaking  talents  or  by 
their  planning  talents  or  by  their  forecasting  talents, 
and  so  forth. 

Some  early  evidence  even  suggests  that  ’!  students 
are  given  opportunities  to  practice  these  talents  or 
processes,  these  talent  processes  are  likely  to  show  more 
transfer  of  training  and  be  more  valuable  in  later  life 
than  will  the  sheer  knowledge  that  they  acquired, 
partly  because  of  the  exploding  and  changing  nature  of 
man’s  knowledge.  We  suspect  that  schools  have  focused 
almost  so  hig'ily  on  knowledge  acquisition  that  much 
of  the  meaning  of  knowledge  has  been  lost  to  students; 
too  often  we  teach  them  the  dead  corpse  of  knowledge 
after  all  the  “life’’  is  out  of  it.  We  now  have  some  fairly 
strong  hints  that  if  we  focus  more  cn  talent  develop¬ 
ment  in  students  than  we  do  in  knowledge  acquisition, 
then  interestingly  enough,  the  students  acquire  more 
knowledge  as  a  by-product  than  when  we  have  them 
focus  so  highly  and  directly  on  its  acquisition. 

1’he  X’s  in  the  first  column  in  figure  1  shows  that 
generally  students  have  developed  primarily  their 
academic  talents  and  have  stayed  within  this  narrow 
band  of  talents  while  they  are  acquiring  subject  matter. 
As  a  breakaway  we  are  doing  all  we  ran  to  learn  how 
to  add  another  column  of  X’s  by  having  them  develop 
their  creative  talents  while  they  are  acquiring  subject 
matter.  We  have  made  similar  progress  with  com 
munication  talents.  We  are  now  starting  to  get  readv 
so  we  can  give  students  experience  in  developing  their 
planning  talents,  their  forecasting  talents,  and  their 
judgmetital  or  evaluative  (decisionmaking)  talents,  ns 
referred  to  earlier. 

All  research  am'  the  classroom  demonstrations  are 
leading  to  the  conclusion  that  hat  is  ner-ded  in  edin  a- 
tion  is  not  ju.st  more  of  the  .same,  hut  instead  to  trv  to 
develop  creative  and  plannirg  and  other  talents.  One  of 
our  demonstration  studies  by  Hutchinson  is  quite  de 
VBstating  to  .s<iine  erlucators.  He  hail  two  comparison 
groups.  One  was  the  typical  c|a.s,srooni  where  students 
are  thought  of  as  learners.  Tlie  ses'ond  tv[>e  of  class¬ 
room  which  was  found  to  tw  quite  a  difTrrrnt  ty[>e  with 


different  people  being  the  star  students;  it  was  one 
where  students  are  thought  of  as  thinkers  rather  than 
merely  learners,  or  absorbers.  So,  we  are  ready  for 
another  type  where  the  students  are  to  be  thought  of 
as  planners,  and  still  another  where  they  are  to  be 
forecasters.  Those  gifted  in  the  learners  classroom  are 
not  those  who  are  gifted  in  the  thinkers  classroom  and 
they  in  turn  are  not  those  who  will  be  gifted  in  the 
planners  classroom  or  in  the  forecasters  classroom.  So 
much  for  my  introduction  to  the  matter  of  talent  identi¬ 
fication  and  r;.Itivation  in  education.  Let  us  now  move 
toward  the  world  of  work. 

Asa  consultant  with  the  Stanford  Research  Institute 
I  have  had  many  challenges  in  dealing  with  several 
groups  of  vice  presidents  in  charge  of  planning.  This 
has  been  a  delightful  experience.  After  working  with 
people  of  all  ages,  1  must  honestly  say  that  I  have 
rarely  run  into  a  group  of  people  more  hungry  for 
information  and  insight,  who  are  more  searching  for 
capabilities  and  techniques  on  how  to  do  things,  than 
this  group  of  vice  presidents  in  charge  of  planning. 

Let  me  explain  w'ith  a  brief  story  which  will  illumi¬ 
nate.  through  perhaps  exaggerate  their  predicament.  It 
goes  like  this:  Whenever  a  person  holds  jealously  unto 
himself  any  activity  as  his  own  sole  prerogative,  all 
other.s  around  him  will  be  deprived  of  experiencing 
whatever  that  activity  is.  In  othe.  words,  whatever 
talents  are  called  for  in  those  activities  will  lend  to 
remain  dormant  in  other  persons  who  are  deprived  of 
participating  in  those  activities. 

When  a  person  is  born  int(^  this  world,  he  has  poten¬ 
tial  planning  talents  that  could  become  active  and  be 
used.  But  he  soon  finds  that  as  an  infant  and  as  a  child, 
fie  does  not  get  loo  much  of  a  chance  in  his  own  home 
to  do  much  planning  because  someone  else  considers 
planning  to  be  their  function  and  duty  and  hold.s  this 
planning  function  Jealously  unto  tbeniselves.  Rut  he 
soon  grows  up  and  finally  he  get.’  a  i  fiance  to  leave  bis 
home  during  the  day  and  go  to  si'hool.  Wiien  he  gets 
in  school,  he  rnav  think,  “Ni.w  1  will  be  able  to  use  my 
planning  talents  and  lliev  will  bw  ome  active  instead  of 
dormant  talents.”  But  he  soon  di.-a'ovcrs  tliat  someone 
else  does  all  the  planning  around  (here.  In  fact  the 
evidence  is  that  fie  is  prohablv  working  under  ex- 
Irrmrlv  riose  siqiervision  in  school  and  is  highly  domi¬ 
nated  lo  the  tea<  her.  In  ahoii!  80  percent  or  so  of  the 
lime,  the  teni  her  is  talking  or  otherwise  dominating  the 
elass.  So  apparently  hr  iloesn’t  get  too  much  of  a 
ehance  to  do  anv  phanning  there. 

.\fter  tie  finishes  s(  hool,  he  gels  a  joh  am!  hires  in  at 
the  l.ottoin  of  the  ladder  and  thinks,  “Now  I'll  get  a 
chanre  to  do  some  pl.anii'ng."  But  siiOdenlv  lie  disi overs 
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that  someone  else  does  all  the  planning  there.  (One  of 
the  audience  suggested  the  next  possibility  to  me.)  So 
I  the  person  decides  to  get  married  and  does  so,  but  he 
;  soon  discovers  that  someone  else  does  the  planning 
i  there,  too.  (Incidentally,  it  has  been  said  that  two 
people  get  married  and  they  become  one.  The  problems 
start  when  they  try  to  decide  which  one.) 

!  If  he  is  a  good  worker,  he  eventually  is  promoted  to 
!  be  a  first-level  supervisor.  Now  he  feels  he  is  going  to 
do  the  planning,  except  he  soon  finds  that  the  first-level 
I  supervisor  doesn’t  do  the  planning.  It  is  apparently 
done  higher  in  the  company.  So  he  continues  to  be  a 
I  good  worker  without  doing  any  planning  and  he  again 

\  is  promoted  upwards.  After  30  or  40  or  more  years  of 

I  his  life  in  programs  where  he  is  deprived  of  all  experi- 
i  ences  in  planning  and  by  having  done  so  well  in  non- 
'  planning  activities,  he  suddenly  is  rewarded  by  being 
,  appointed  vice  president  in  charge  of  planning.  And 

I  believe  me,  he  and  others  like  him  are  hungry  for 

!  information  and  skills  in  planning. 

\  In  our  educational  chart  (fig.  II-l)  let  me  give  you 

(  an  example  of  how  we  might  fill  out  one  of  those  X’s  for 

1  planning  talents  in  students  while  they  are  acquiring 

I  subject  matter.  I  will  deal  with  a  known  rather  than 

i  unknown  area  of  subject  matter,  since  teachers,  by  and 

I  large,  are  not  yet  prepared  to  deal  in  areas  either  at  the 

}  fringe  of  knowledge  or  out  beyond  where  no  one  yet 

j  knows  the  answer.  One  possibility  is  to  give  students  a 

I  map  of  the  physical  and  biological  resources  of  our 

J  Nation  without  any  markings  about  the  population 

1  distribution.  Show  them  where  the  mountains,  forests, 

I  rivers,  the  fertile  ground,  and  all  the  natural  resources 

j  are.  'Fhen  ask  them  to  plan  a  nation,  saying:  “Where 

i  would  the  people  start  cities  and  where  would  they  be 

i  doing  different  kinds  of  things.”  Let  the  students  plan 

I  where  the  people  would  be  and  what  kind  of  activities 

^  would  occur  in  which  places  in  the  nation.  The  teacher 

I  would  keep  secret  from  them  where  these  things  have 

I  already  occurred.  After  they  map  out  the  whole  thing, 

;  chances  are  that  they  will  be  eager  to  check  up  to  see 

!  to  what  degree  the  nation  had  really  developed  accord- 

!  ing  to  their  plan.  We  suspect  that  they  will  be  much 

1  more  interested  in  acquiring  this  information  than  they 

S  would  be  if  we  had  just  dished  out  all  the  answers  to 

them  as  is  often  customary,  on  a  silver  platter  to  learn 
I  (memorize).  Then,  after  you  did  that,  you  could  lead 
!  them  into  the  unknowns  which  will  emerge  from  popu¬ 
lation  explosions  by  having  them  make  long-range 
i  forecasts  through  these  instructions:  Now,  where  is  the 

1  rest  of  the  expanding  population  going  to  go?  What 

j  are  we  going  to  do  now  in  light  of  man’s  latest  knowl- 

I  edge — in  terms  of  what  he  can  now  do  in  the  way  of 


utilization  of  resources  and  new  means  of  production  of 
energy  and  so  on?  Where  will  the  populations  of  the 
future  be  located  and  distributed? 

This  gives  you  an  example  of  how  they  could  both 
plan  and  check  up  on  their  plans  from  the  past  and  in 
the  second  case,  how  they  could  map  out  a  long-range 
forecast  or  plan  for  the  future.  We  could  also  check  and 
see  if  any  of  their  future  plans  were  any  better  than 
the  plan  of  the  moment  of  our  experts.  Maybe  they  can 
think  more  freely.  There  is  some  evidence  that  when 
students  get  into  school,  they  learn  how  not  to  be 
inquiring  as  they  go  further  through  school.  Maybe 
the  earlier  that  we  get  them  started,  the  better  they  can 
use  their  natural  planning  and  forecasting  talents 
without  built-in  restrictions  from  their  negative  life 
experiences  and  training.  No  doubt,  all  of  you  could 
suggest  many  more  planning  tasks  that  could  be  used 
in  testing  and  training  the  planning  and  the  fore¬ 
casting  talents  of  students.  We  should  soon  need  such 
suggestions. 

As  we  look  at  these  multiple  types  of  talent,  there  is 
a  possibility  that  the  really  top  persons,  the  “true 
experts,”  are  those  who  have  a  high  profile  not  only  in 
planning  talents,  but  also  in  one  or  more  of  these  other 
pertinent  talent  areas,  too,  such  as  creativity  (including 
flexibility)  and  communicating  and  decision  making, 
and  in  effective  integration  of  these  various  types  of 
talent  activities.  So  particular  talent  combinations 
could  produce  a  very  interesting  approach  for  studying 
the  selection  of  experts. 

I  think  we  should  realize  that  whether  we  have  used 
good  experts  or  poor  experts,  in  any  event  we  do  have 
a  plan  and  this  plan  can  and  usually  does  shape  the 
future  to  some  large  or  small  extent.  It  does  affect 
people  and  most  of  them  tend  to  work  within  the  plan, 
not  outside  the  plan,  and  the  plan  is  imposed  almost 
as  a  restriction  upon  them.  So  the  future  is  partly 
shaped  by  a  plan,  whether  the  plan  was  produced  by 
people  with  a  high  degree  of  planning  talent  or  not. 

In  the  same  vein,  I  am  told  that  the  President  of  the 
United  States  can  always  guarantee  that  he  can  get 
advice.  What  he  cannot  guarantee  is  that  he  can  get 
wise  advice.  So  you  see  some  of  the  talents  that  the 
psychologists  of  the  past  have  ignored  and  postponed 
for  the  future  have  been  some  ojf  these  other  high-level 
talents,  like  exist  in  the  wise  men  or  in  those  that  have 
great  foresight  and  vision  and  prophecy  ability  and 
judgmental  ability,  creative  ability,  and  so  on.  We  are 
starting  to  move  more  vigorously  into  plowing  these 
fertile  talent  areas  that  were  largely  ignored  until  a 
decade  ago. 

Man  has  a  tendency  to  narrow  a  problem  down  so  he 
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can  cope  with  it,  so  some  people  often  work  in  a  very 
narrow  frame.  Certain  persons  may  have  fewer  ideas 
and  can  only  juggle  fewer  ideas  and  less  information 
and  less  variableo  in  their  minds  at  one  time.  This  is 
one  of  the  poiential  bases  for  differences  in  people’s 
creativity  and  it  may  be  true  about  differences  in 
ability  to  plan  and  also  to  make  long-range  forecasts. 
If  a  person  deals  with  fewer  variables  at  one  time,  his 
resulting  plan  or  forecast  may  be  equally  restricted  by 
his  lack  of  dealing  with  all  pertinent  variables.  Maybe 
those  who  are  best  are  those  who  can  juggle  and 
manipulate  many  more  variables  at  one  time,  and 
eventually  work  with  them  all  at  once  and  then  sud¬ 
denly  pull  them  together  into  a  plan  or  a  forecast. 

One  incident  in  the  history  of  the  behavioral  sciences 
in  this  Nation  should  be  recalled  in  which  they  made 
some  forecasts.  After  the  forecasts,  they  got  together 
and  took  stock  nationally  and  professionally  about  this 
business  of  forecasting.  This  was  the  time  when  the 
newspapers  erroneously  printed  that  Thomas  Dewey 
had  been  elected  President  of  the  United  Stales.  One 
bit  of  advice  that  the  forecasters  from  the  behavioral 
science  gave  themselves  was  that  it  would  be  quite 
possible  to  retreat  into  only  those  areas  of  forecasting 
where  there  is  no  severe  checkup  on  their  work  and 
where  they  still  would  be  able  to  make  a  good  living. 
Or  they  could  carefully  avoid  making  anything  but  the 
longest  range  forecasts  which  would  be  so  far  ahead 
that  they  would  be  safe  since  few  of  their  generation 
would  still  be  around  alive  to  check  up  on  them.  In¬ 
stead,  they  argued  to  stay  in  the  election  and  other 
tough  forecasting  arenas  where  they  would  continue  to 
encounter  severe  checkups  on  their  forecasts. 

A  great  deal  of  our  research  effort  is  spent  on  the 
criterion  problem  of  measuring  the  total  complexity  of 
career  performances  and  accomplishments  of  profes¬ 
sional  and  other  people.  For  example,  we  obtained  50 
measures  of  what  research  enter  scientists  had  con¬ 
tributed  in  their  career.  We  have  80  measures  of  what 
physicians  accomplish  in  their  career  and  we  are  told 
we  didn’t  cover  en<nigh,  st)  we  have  a  new  project  to 
move  further  into  measuring  their  (rerforniances  and 
achievements.  We  have  a  somewhat  different  set  of  80 
measures  to  date  for  medical  faculty  than  we  do  for 
s|»ecia!ists  in  medicine,  and  also  a  particular  set  of  80 
spcciailv  for  gener.al  [iraclitioners.  Thus,  wr  are  strug¬ 
gling  with  a  great  complexity  of  luiman  perlorniances 
and  human  characteristics  in  aii\  given  profe.ssioi:. 

Another  result  that  we  always  find  when  we  study 
human  lieings  is  that  there  arc  alw.iy.s  individual  differ- 
erices  and  often  they  can  he  great  ditfrrcnces.  So  I  ani 
persuaded  that  we  will  find  great  complexities  in  the 


performances  plus  *arge  individual  differences  in  the 
various  planning  and  forecasting  activities.  I  felt  that 
Dr.  Dalkey  presenttal  some  evidence  on  this  point  of 
great  individual  diiferences.  In  a  lot  of  his  group 
studies,  it  might  be  quite  possible  to  tease  out  studies 
of  individuals  as  well  as  of  groups.  Perhaps  ways 
should  he  found  for  some  of  my  research  staff  to  team 
up  with  Dr.  Daikey’s  group  so  that  findings  on 
individual  differences'  can  also  be  extracted  from  his 
group  studies. 

Another  feature  in  which  we  are  interested  is  what 
would  he  called  variability  within  a  person  from  one 
lime  to  another.  This  variability  is  in  contrast  to  what 
we  are  most  often  seeking,  namely  consistency  and 
reliability  within  a  person.  In  some  cases,  variability 
within  a  person  '.night  alternately  be  described  as  flexi¬ 
bility  instead  of  'nconsistency  or  unreliability  within  a 
person  from  one  time  to  another  t'me.  Some  of  Thur- 
stone’s  last  projects  before  his  death  dealt  with  psycho¬ 
physical  me;  sures  of  what  might  prove  to  be  important 
temperamental  tra'ts  such  as  flexibility  in  individuals. 
1  noted  in  listening  to  Daikey’s  comments  that  we  have 
a  lot  of  psychophysical  techniques  which  could  be 
applied  to  the  kinds  of  things  that  are  now  occurring 
in  his  on-going  planning  and  forecasting  activities. 

When  groups  of  p.'Kiple  are  making  judgments  and 
we  see  that  they  differ  individually,  one  of  the  things 
we  might  ask  is  Should  we  weigh  their  opinions  equally 
or  should  we  give  them  some  differential  weight?  Some 
might  facetiously  say,  “One  way  we  can  give  them 
differential  weight  is  to  give  greater  weight  to  those 
who  had  higher  school  grades  or  who  have  higher 
IQ  scores.”  But.  as  the  previous  speaker  indicated,  it 
is  amazing  how  their  talents  in  one  area  are  not  par¬ 
ticularly  related  to  their  talents  and  performances  in 
another.  This  is  our  finding,  too.  Good  things  go  to¬ 
gether  slightly,  and  not  by  any  means  synonymously. 
This  Is  indicated  by  a  little  hit  of  overlap  between 
circles  sketched  to  measure  talents,  with  only  10  percent 
overlap  between  two  talent  circles — and  you  know  10  or 
less  percent  is  not  much  overlap.  With  good  things 
going  together  only  slightly,  we  must  avoid  or  be  very 
cautions  about  weighting  according  to  what  mav  lie  an 
niin.ist  irrelevniit  characteristic.  For  example,  we 
should  trv  to  avoiii  lettina  the  greatest  weiglit  he  given 
to  the  jxTson  who  talks  the  most  on  the  topic  in  the 
group.  Just  the  fact  that  he  talks  the  most  does  not 
mean  he  is  the  best  planner  or  the  liest  forecaster  or 
the  best  judge  or  the  one  with  the  most  creative  ideas 
or  with  the  highest  quality  of  creativeness  of  ideas. 

1  liotieslly  have  the  impression  that  there  are  verv, 
very  few  psychologists  who  are  worki.ig  in  these  im 
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portant  live  world-of-work  planning  and  forecasting 
situations.  Many  others  would  defend  themselves  by 
saying  such  things  as,  “You  are  playing  with  prob¬ 
lems  that  are  too  applied  or  too  uncontrolled,”  and  so 
on.  We  almost  walked  alone  when  we  initially  started 
to  study  the  world-of-work  performances  of  scientists, 
of  physicians,  of  medical  faculty,  and  of  nurses.  Almost 
everyone  told  us  that  it  couldn’t  be  done,  but  we  tried 
npyway.  One  of  the  deficiencies  of  our  profession  is 
this  overstrong  tendency  to  continue  to  work  only  back 
in  the  old  areas  and  not  being  very  willing  or  able  to 
move  into  new  untouched  areas.  The  notion  that  the 
field  is  ready  to  be  plowed  is  a  good  notion.  There  are 
lots  of  old  fields  now  being  plowed,  but  some  of  the 
potentially  most  fertile  ones  are  oftimes  the  ones  that 
are  not  being  touched  at  all. 

We  do  have,  I  think,  a  lot  of  techniques  that  might 
be  effective  in  helping  us  delve  into  these  new  fields. 

I  sense  in  all  of  Dalkey’s  cases,  that  we  could  obtain 
multiple  scores  on  the  performances  of  each  of  these 
people  on  an  individual  basis.  Most  of  the  few  tests  in 
the  field  of  planning  so  far  are  largely  restricted  to 
paper-and-pencil  (schoollike)  tests.  From  our  research 
experiences  with  situational  and  other  tests,  we  could 
also  set  up  situations  of  various  kinds  of  planning 
activities  from  which  we  could  obtain  multiple  scores 
as  an  effective  means  to  analyze  the  behaviors  of  ex¬ 
perts  in  practice.  In  our  recent  study  of  Peace  Corps 
abilities  we  built  some  live  situational  tests  and  found 
they  called  for  a  lot  of  abilities  (or  talents)  that  paper- 
and-pencil  tests  could  not  or  at  least  did  not  get  at. 
These  abilities  tend  to  be  the  live  complex,  world-of- 
work  situations  and  activities  because  they  do  not 
overlap  much  with  the  academic  world  where  most  of 
the  psychological  research  work  on  talents  has  been 
done  so  far. 

In  our  work  on  scientists,  self-ratings  on  creativity 
were  very  good,  compared  to  other  scores,  and  we  could 
get  them  almost  Instantaneously.  Our  hunch  from  this 
is  that  self-ratings  are  very  promising,  but  with  one 
most  important  condition:  people  have  to  be  given 
many  opportunities  to  experience  tbeir  talents  on  which 
we  eventually  want  their  self  ratings.  For  example,  if 
the  students  who  started  in  my  hypothetical  nursery 
school  were  to  practice  their  planning  talents  and  to 
continue  such  practicing  throughout  their  school  and 
work  careers,  by  the  time  they  got  to  where  they  were 
contenders  for  vice  presidents  in  charge  of  planning  or 
vice  presidents  in  any  other  kind  of  activity,  these 
people  would  be  able  to  give  you  quite  valid  self-ratings 
on  their  planning  talents. 

I  also  sensed,  in  Dalkey’s  studies,  that  irrelevancy 


and  noise  in  the  discussions  were  coming  out  from  the 
participants.  I  would  bet  on  tremendous  individual 
differences  in  how  much  people  contribute  to  the 
irrelevancy  and  noise  because  some  have  a  strong 
screening  process  before  they  speak,  while  others  are 
almost  nonstop  talkers  and  at  times  have  no  barriers 
on  what  flows  out.  (There  are  even  suspicions  that  some 
people  talk  maybe  even  faster  than  they  think,  although 
the  reverse  is  most  usual.)  To  the  degree  that  this 
irrelevancy  is  going  on,  the  ability  to  select  relevant 
from  irrelevant  might  be  one  of  the  important  things 
in  the  group  planning  activity  and  therefore  one  of  the 
things  in  the  profile  of  the  high-level  talented  people 
upon  which  we  should  focus. 

Rethinking  is  also  interesting  when  it  occurs  because 
revision  ability  in  our  communication  studies  was  the 
best  single  device  we  found  for  predicting  performance 
in  world-of-work  communication  situations.  You  can 
nearly  always  rework  your  communications  because 
they  are  usually  somewhat  ambiguous  the  first  time.  If 
you  don’t  believe  it,  start  expressing  yourself  by  writ¬ 
ing  a  long  program  to  a  computer  and  see  if  you  can 
ever  get  it  completely  clear  the  first  time  without 
needing  any  debugging.  So  this  rethinking  is  one 
instance  in  which  we  are  asking  for  variability  within 
a  person.  Variability  is  likewise  needed  in  flexibilities, 
including  the  ability  to  get  new  information  and  adjust 
oneself  and  have  a  change  in  reactions  and  plans  and 
forecasts  in  light  of  the  new  information.  Just  plain 
wjrambling  is  another  form  of  variability  in  perform¬ 
ance  that  may  appear  to  be  unplanned  and  disorgan¬ 
ized,  but  as  in  the  case  of  quarterbacks  it  can  at  times 
eventually  prove  to  be  quite  effective. 

A  potential  complication  is  that  some  people  have 
pretty  well  convinced  themselves  that  they  don’t  want 
to  play  the  revision  or  rethinking  approach  because 
when  they  take  a  test  their  first  overall  impression  is 
often  better  than  their  rethinking  impression.  When 
they  change  their  answers,  they  say  they  run  into 
trouble.  We  should  recognize  this  possibility,  too,  in 
planning  and  forecasting.  If  a  person  already  had  a 
first  overall  impression  before  he  got  into  the  group 
discussions,  sometimes  his  initial  impression  had  more 
validity  than  one  he  might  change  to  after  he  looked 
in  detail  at  all  the  relevancies  and  irrelevancies  and 
then  mixed  them  all  up  together  as  the  basis  for  his 
change. 

Pressures  in  a  group  may  be  worth  studying  to  see 
who  produpes  the  group  pressures,  who  gives  in  to  the 
group  pressures,  and  how  these  pressures  are  related 
to  the  final  actions  of  the  group.  I  do  not  know  what 
would  happen  in  the  area  of  planning  but  in  the  area 
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of  creativity,  the  people  who  do  not  give  into  the  group 
pressures,  and  who  are  willing  to  tell  the  truth  and 
report  it  as  they  see  it,  independently  if  necessary  of 
the  pressures.,  are  the  most  creative.  So  I  suspect  that 
two  or  three  creatives  in  the  group  might  move  farther 
away  and  diverge  from  the  consensus  rather  than  con¬ 
verge  when  group  pressures  emerge.  I  am  wondering 
whether  maybe  some  of  our  most  expert  planners,  those 
highly  gifted  in  planning,  might  also  show  some  of 
these  rugged  individualistic  tendencies  when  working 
with  groups  in  the  area  of  their  own  greatest  talent.  I 
doubt  if  anyone  knows;  it  would  be  interesting  to  find 
out. 

In  our  communication  ability  studies,  we  felt  that 
much  of  group  activity  could  be  accounted  for  on  an 
individual  basis — not  all  but  much  and  probably  the 
majority  of  the  group  phenomena.  If  we  knew  each 
jjerson’s  individual  profile  of  communication  talents, 
we  felt  we  could  account  for  a  lot  of  what  would  happen 
when  groups  of  such  people  got  together  because  of 
the  individual  characteristics  of  each  in  terms  of  their 
communication  talents.  I  would  suspect  that  we  might 
also  find  some  of  that  in  the  area  of  planning:  That 
much  of  the  group  behavior  could  be  accounted  for  if 
we  knew  each  individual’s  profile  of  the  planning  and 
other  relevant  or  irrelevant  talents  elicited  in  the 
groups.  Then  there  would  be  some  other  distinctly 
group  factors  above  and  beyond  the  individual  (non¬ 
group)  ones  that  would  remain  to  be  explained,  but  I 
suspect  these  remaining  aspects  would  usually  entail 
only  a  minor  part  of  the  total  phenomenon. 

Since  all  of  you  are  pioneering  this  new  area,  may  I 
comment  about  problems  you  may  encounter  based 
upon  experiences  I  have  had  in  trying  to  help  pioneer 
in  the  field  of  creativity.  One  idea  came  to  me  when  I 
was  working  in  the  National  Research  Council,  whose 
full-time  business  is  research  counciling.  I  sensed  that 
there  was  an  in-group  for  each  specific  science  area 
who  wielded  an  influence  as  well  as  some  control  on 
the  progress  in  their  area.  I  wondered  what  the  chances 
were  of  a  new  person  ever  getting  invited  into  the 
in-group  council  meeting  and  if  he  did  get  invited,  what 
chance  would  he  have  to  contribute  to  the  short  report 
of  their  meeting  that  went  to  the  outer  circle.  Further¬ 
more,  if  he  were  invited  but  proved  to  be  a  young  up¬ 
start  who  made  some  comment  that  didn’t  fit  the  ideas 
of  the  leaders,  those  who  currently  owned  the  field,  so 
to  speak,  what  chance  would  he  have  of  getting  in  his 
idea  and  having  it  received  by  the  inner  circle  and  also 
included  in  the  writeup  sent  to  the  outer  circle.  On  these 
points  I  think  you  have  some  protections  in  this  8eri» 
of  two  conferences:  First,  you  are  rotating  members 


and  speakers  to  some  degree ;  secondly,  you  are  record-  | 
ing  the  discussions,  so  that  they  are  published  instead  j 
of  screened  out  by  the  inner  circle;  and,  thirdly,  you  j 
are  using  cross  fields  persons,  such  as  those  present,  | 
who  will  provide  a  further  protection  against  the  tyj^  | 
of  errors  and  distortions  described  above.  At  least,  this  .  ; 
has  been  my  experience.  ' 

In  turning  to  the  planning  processes  within  a  person, 
including  the  input  processes,  the  central  processes,  and 
the  output  processes,  the  cycle  can  be  repeated  in  terms 
of  checking  up  on  your  plans  and  your  forecasts  and 
revising  them.  In  other  words,  you  can  repeat  the  input, 
central,  and  output  processes  in  doing  planning  work. 

I  suspect  that  the  input  can  be  mighty  important.  The 
input  could  include  the  whole  past  life  history  and  ex¬ 
periences  of  a  person  and  all  the  immediate  things  that 
are  coming  in  that  do  or  could  bear  on  what  he  is  now 
doing.  I  suspect  this  total  input,  both  past  and  present, 
might  be  much  more  important  than  we  have  given  it  ; 
credit  for.  The  total  reservoir  of  inner  resources  of  a 
person  would,  at  least  potentially,  be  available  to  draw 
upon  if  a  person  knew  how  to  do  so  and  had  the  central 
processes  working  well  to  use  his  large  inner  library  of 
resources.  The  central  processes,  I  suspect,  can  be  very 
complex  and  the  studies  that  have  been  done  on  plan¬ 
ning  indicate  that  the  more  complex  the  planning  ac¬ 
tivities,  the  more  complex  the  set  of  talents  that  are 
needed  to  work  in  the  activity.  Since  some  of  our  tests 
are  very  simplified  tests,  we  would  have  to  use  a  collec¬ 
tion  of  these  tests  to  get  at  all  aspects  of  tliese  central 
processes.  The  output  (expressional)  processes,  too, 
can  be  full  of  interesting  problems  in  each  area  of 
human  activity,  including  planning  and  forecasting. 

Let  us  examine,  for  a  moment,  the  input  processes 
together  with  the  flexibility  of  the  person  engaged  in 
planning.  Can  a  person  continue  to  listen  and  receive 
new  information  as  well  as  he  could  when  he  first 
started  to  plan  after  he  has  progressed  some  distance 
toward  making  a  plan  or  even  after  he  has  crystallized 
a  plan? 

When  a  person  an3/or  a  group  has  completed  a  plan, 
to  what  degree  have  they  fully  and  appropriately  used 
the  available  inputs.  Are  the  facts,  ideas,  suggestions, 
and  even  something  so  far  out  as  suggested  plans  of 
others  fully  listened  to  and  utilized?  One  of  the  human 
tendencies,  I  fear,  is  that  whenever  someone  comes  up 
and  makes  a  suggestion  to  a  person  in  a  crucial  posi¬ 
tion  in  the  organization  who  is  responsible  for  plan¬ 
ning,  that  person  in  the  status  position  may  make  a 
human  reaction  such  as:  “That’s  my  business,  not 
yours,  to  make  the  plan.  It’s  none  of  your  business.’’ 

If  the  suggester  goes  too  far  by  using  his  own  talents  to 
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I  form  his  suggestions  into  a  suggested  plan  and  com- 

I  municates  only  his  suggested  plan,  the  official  planner, 

j  of  course,  may  feel  threatened  or  even  feel  the  sug- 

gester  is  trying  to  sharpshoot  him  out  of  his  position. 
So  this  kind  of  complexity  can  occur  in  the  input 
processes. 

Apparently  one  of  the  most  difficult  things  for  any 
supervisor  to  listen  to — and  it  is  hard  for  a  suptervisor 
to  listen — is  to  listen  to  a  subordinate,  especially  if  this 
subordinate  has  a  highly  creative  new  idea.  Now,  maybe 
it  is  equally  difficult  for  a  supervisor  to  listen  to  an 
I  extremely  high  quality  plan  from  below.  Maybe  one  of 

j  the  problems  is  that  some  of  the  best  ideas,  best  infor- 

j  mation,  best  suggestions,  and  maybe  the  best  plans  may 

I  occur  in  the  organization  but  in  the  wrong  minds  in  the 

i  organization — in  the  wrong  places  in  the  organization 

I  chart,  perhaps  at  the  grassroots.  Hov/  to  get  the  sug- 

1  gested  information,  ideas,  and  plans  from  there  through 

I  a  set  of  human  communicators  to  the  right  place  in  the 

(  organization  and  have  them  be  transmitted  without 

erroneous  distortion  and  be  accepted  effectively  at  all 
stages,  including  the  last  one  where  the  responsibility 
resides,  will  be  found  to  be  one  of  our  major  problems, 
I  believe,  in  this  planning  area. 

Other  questions  are:  Who  is  included  in  the  plan  and 
what  input  is  included  in  the  plan?  And  who  is  ex¬ 
cluded  from  the  plan  and  what  is  excluded  from  the 
plan?  The  organization  may  have  a  screening  process 
here  in  which  it  is  screening  out  those  with  the  higher 
level  planning  talents  or  screening  out  or  partially  dis¬ 
torting  the  better  ideas  and  better  facts.  Thus,  some  of 
the  best  potential  information  and  talent  resources  in 
the  organization  may  be  organized  out  during  the  plan¬ 
ning  processes. 

Let  me  illustrate  these  points  in  terms  of  our  new 
field  called  architectural  psychology,  in  which  the  archi- 
I  lects  design  environments  and  the  psychologists,  study 
the  reaction  of  people  to  these  environments.  As  the 
initial  psychologist  on  this  team,  I  have  realized  that  a 
lot  of  the  activities  of  architects  fall  into  the  field  of 
designing  and  planning.  Our  architectural  psycholo¬ 
gists,  including  our  graduate  students,  have  made  some 
plans  about  hospitals  and  we  have  run  into  the  above 
problems.  We  go  to  multiple  sources  for  information 
that  might  be  pertinent  to  the  future  plan.  As  we 
worked  in  hospitals,  we  have  found  at  times  that  it  was 
difficult  to  get  much  usable  information  from  physi¬ 
cians.  Our  work  with  psychiatrists  uncovered  that  they 
did  not  easily  become  very  explicit  in  terms  of  the 
architectural  problems  and  secondly,  they  showed  us 
almost  anything  but  agreement  among  themselves.  The 
further  down  in  the  organization  that  we  went  for  infor¬ 


mation,  almost  the  better  the  information  we  were  get¬ 
ting  from  these  usually  least  expected  sources.  Some  of 
our  best  information  quite  frequently  came  from  the 
lowest  people  in  the  hospital  organization,  from  nurses’ 
aides  and  from  maintenance  people.  They  were  the  ones 
who  gave  us  considerable  information,  but  we  suspect 
that  some  architects  won’t  listen  to  these  people  or  per¬ 
haps  really  the  architects  do  not  get  a  chance  to  listen 
to  them.  Usually  they  only  work  with  their  client  and 
their  client  is  up  at  the  top  and  perhaps  the  client  does 
not  prove  to  be  a  good  representative  for  all  the  other 
persons  in  the  organization  and  does  not  listen  to  them 
or  even  realize  he  should  listen  to  them  so  he  can  repre¬ 
sent  them  well  and  make  their  contributions  available 
to  the  architect. 

Some  of  these  problems  may  stem  from  the  fact  that 
architects  often  do  not  have  a  “live”  client  in  their 
training,  but  only  their  teacher.  So  their  tendency, 
when  they  get  into  practice,  is  to  practice  like  they  were 
trained  to  do,  except  to  do  without  their  teacher.  There¬ 
fore,  an  occasional  client  may,  at  times,  almost  have  to 
fight  his  way  into  the  architect’s  creative  and  planning 
processes  to  make  his  points  (and  I  am  exaggerating  a 
little  again  to  make  my  point) . 

The  planner  also  has  the  problem  of  understanding 
the  information  he  is  getting,  so  there  is  a  conununica- 
tion  process  here  and  also  one  of  integrating  it  after  he 
gets  all  of  it.  In  architecture  again,  one  of  the  things 
that  comes  out  of  the  communication  process  is  a  set  of 
drawings  or  blueprints,  which  represent  a  plan.  We 
have  just  realized  it  is  not  only  ».  plan — it  is  a  com¬ 
munication  device,  too,  between  a  second  and  third 
person  in  the  chain.  The  architect  produces  this  plan  for 
the  sake  of  the  third  person  in  the  chain,  namely,  the 
production  or  construction  pe^iple.  After  the  client  has 
communicated  in  terms  of  words  with  the  architect,  a 
problem  arises  if  the  client  Cinnot  understand  the  set  of 
drawings  which  are  the  plan.  The  architect  has  pro¬ 
duced  a  visual  plan  for  the  production  people,  but  the 
client  may  often  have  a  problem  of  comprehending  the 
complex  spatial  drawings. 

To  handle  some  of  these  communication  problems  in 
design,  a  project  on  our  campus  is  planning  how  to  use 
a  computer  in  graphic  outputs  in  architectural  design. 
The  computer  can  potentially  handle  a  lot  of  input  in¬ 
formation,  too,  and  can  double  check  things  that  are 
happening  so  that  it  can  do  a  lot  of  the  structural  engi¬ 
neering  and  other  technical  work.  Then  the  architect 
will  be  free  to  spend  a  higher  percentage  of  his  time  in 
creative  planning  and  designing  and  communicating  at 
all  stages  with  the  client.  Potentially  he  can  plan  and 
design  a  higher  percentage  of  buildings  and  the  man- 
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made  space  that  men  will  occupy  in  the  future.  Thus, 
the  computer  can  also  be  a  communication  device  to 
clarify  the  plan  to  the  client  and  can  give  feedback  to 
the  client,  to  include  him  in  more  throughout  all  stages 
in  the  planning  so  he  can  ask  for  modifications  where 
necessarv  before  the  plan  comes  out  in  its  final  form  of 
solid  cement  walls. 

In  terms  of  city  planning,  someone  recently  said  that 
“cities  are  people,  and  not  just  buildings.”  Again  this 
means  that  someone  in  the  past  has  not  given  enough 
of  their  thoughts  to  the  people,  or  has  not  thought 
enough  of  the  people,  so  something  has  been  neglected 
or  overlooked  in  the  planning  as  far  as  the  people  are 
concerned.  Yesterday  one  of  my  students  said,  “When 
we  plan  for  the  heart  of  our  cities,  which  might  become 
the  slum  and  crime  delinquency  areas,  we  must  con¬ 
sider  what  the  place  will  be  like  at  night  and  how  many 
dark  alleys  we  have  produced  both  inside  and  around 
apartment  buildings  as  well  as  outside  in  other  places.” 
This  aspect  of  planning  could  have  easily  been  over¬ 
looked  in  the  past,  leading  to  plans  which  have  actually 
facilitated  crimes. 

There  is  a  plan  for  cities  of  the  future  called  the 
Model  Cities  Plan.  The  designers  have  been  very  proud 
and  pleased  with  this  plan,  except  that  during  the  riots 
last  summer,  a  statement  was  made  by  some  of  the  riot 
leaders  that  they  dismissed  this  plan  as  worthless.  In 
effect,  they  said  that  it  was  a  white  man’s  plan  instead 
of  a  black  man’s  plan  (even  though  the  planners  argue 
that  Negroes  have  had  some  representation).  They 
want  to  own  th<  ir  own  communities,  namely,  much  of 
the  heart  of  the  cities.  When  these  slum  places  are  reno¬ 
vated,  they  also  want  to  be  involved  in  the  renovation 
process  by  being  crucial  parts  of  the  construction  team, 
and  p.srt  of  those  who  are  making  a  profit,  rather  than 
having  a  white  man’s  construction  team  come  in  and 
largely  exclude  them  from  the  work  force.  They  ma) 
also  have  the  feeling  that  they  have  })ecn  included  out 
in  the  planning,  in  spite  of  the  fact  that  they  arc  the 
ones  who  are  going  to  be  the  tenants.  People  who  move 
into  buildings  as  tenants  art  often  probably  not  even 
»cle<'ted  as  tenants  when  the  buildings  were  being 
planned.  So  we  have  a  real  planning  problem  there. 

.Another  kind  of  prohle.'n  whi<  h  is  hapjrening  in  the 
citv,  is  a  grassroots  niovenient,  which  ran  have  a  lot 
of  healthiness  to  it.  Hie  people  in  the  slum  area  want  to 
renovate  their  own  area  f<'r  thrmsehes  '  istead  of  f>eins 
wipes!  out  and  having  to  relix-atr  eiscwhrre  while  some 
new  Iniiidings  go  up  and  some  other  groups  of  iwople 
move  into  this  nesri;,  huiit  area.  One  asfteei  is  that  the 
rltv  fathers  who  have  Ikw-ti  successful  in  handling  all 
these  previous  communitv  problems  mav  now  Iw-  plan 


ning  what  they  are  going  to  do  next  with  the  area.  But 
at  the  same  time  other  plans  are  emerging  at  the  grass¬ 
roots  level  and  the  question  arises:  What  is  going  to 
happen  to  these  two  competing  plans  and  are  they  ever 
going  to  become  synchronized  together  somehow  to 
become  only  one  plan? 

In  our  work  for  the  Peace  Corps,  we  have  tried  to 
learn  how  to  select  Peace  Corps  volunteers  who,  when 
they  will  go  overseas,  will  not  be  the  highly  talented 
and  display  their  great  talents  overseas  to  become  the 
great  performers  themselves.  Instead  we  wanted  to  se¬ 
lect  and  train  them  to  be  subtle  catalysts  so  that  the 
host  country  people  develop  their  own  talents  and 
thereby  become  the  great  performers.  We  wanted  our 
Peace  Corps  volunteers  to  spark  the  people  overseas  to 
become  creative  themselves,  to  learn  how  to  sense  prob¬ 
lems.  how  to  create  plans  themselves,  and  to  follow 
through  and  implement  those  plans.  This  may  be  an 
interesting  challenge  in  city  planning  and  broad  plan¬ 
ning  of  the  future:  how  can  we  go  in  and  be  the  spark¬ 
plugs  to  help  the  other  people  make  their  own  plans 
instead  of  doing  it  ourselves  and  then  imposing  the 
plans  on  them? 

In  the  magazine  industry,  the  cardinal  sin  is  to  follow 
the  plan  right  out  the  window.  I  heard  this  when  I  was 
on  a  14th  floor  where  it  had  real  meaning  to  me.  So 
sometimes  there  has  to  be  some  continuous  checkup 
with  potential  revisions  'n  the  plan.  In  research  there 
are  times  when  it  is  wise  to  have  a  good  tight  research 
plan  and  other  times  when  \i  is  not  wise  to  be  so  locked 
in.  Sometimes  in  power  struggles  a  way  that  some 
people  get  control  over  other  people  is  to  build  a  plan 
with  the  ulterior  motive  of  control  and  then  try  to  hold 
the  other  people  to  it.  But  the  others  may  not  want  to 
slavishly  f  diow  their  [dan. 

The  issue  in  question  i.-^  the  tentativeness  in  planni.ig 
and  the  capability  of  ,)eople  to  be  tentative  in  their 
planning  or  in  thcir  following  of  a  plan.  This  mav  he 
highly  important  for  leailers  to  be  capable  of  piroducing 
only  a  tentativi  [dan  and  really  function  .so  it  is  a 
tentative  instead  of  a  firmlv  crvstali.red  plan.  Quite 
obviously,  any  needed  revision  that  rnav  arise  is  more 
likely  to  hr  welcomed  if  the  [dan  is  widely  considered 
to  be  triitiitive.  In  contrast,  it  i"  v  be  a  verv  difficult 
thinu  to  tincrv  stallize  a  plan  that  is  highly  crystallized. 

RrsTTil  rrsea  h  on  human  talents  mentions  divergent 
an<l  convergrnt  talents.  I'tie  tlivrrgent  tvpie  invidves 
various  hoiling  u|i  ami  ex[iamliin^  [Uocesses  whereas 
convrijent  talent  rntads  hoiling  down  and  rrshiction 
[iriM-esscs,  foensing  and  zeroing  in  and  converging 
toward  a  solution  \  strong  armim.-nt  .  an  br  made  that 
the  twst  [ilanniiig  mav  be  lii  i  •'!  gent  rather  than  conver. 
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gent.  Or  maybe  the  best  planning  alternates  as  follows: 
converges,  diverges,  converges,  diverges,  converges,  di¬ 
verges.  The  need  is  to  find  persons  who  are  flexible  in 
their  planning,  who  can  alternately  converge  and  di¬ 
verge,  and  open  up  for  further  information  by  uncrys¬ 
tallizing  the  plan  to  some  degree  and  then  converge 
again.  In  this  way,  the  planners  are  always  alert  and 
can  modify  the  plan,  and  thus  have  some  degree  of 
tentativeness,  but  will  also  have  a  plan  ready  that  they 
can  follow,  if  necessary,  at  any  given  time. 

Human  beings  are  often  not  very  good  at  this  kind  of 
flexibility.  I  used  this  technique  on  persons  in  an  or¬ 
ganization  that  was  highly  crystallized  in  its  plans,  as 
the  best  approach  to  use  to  deal  with  them.  I  jvould  say, 
“OK,  we  have  completed  a  plan  today  that  we  can  use 
and  can  have  ready  for  use,  but  now  let’s  reopen  it  and 
rethink  it  again.”  It  was  difficult  for  these  people,  the 
powers  that  be,  to  allow  us  to  do  this.  Yet  i;  was  the 
best  chance  I  had  to  get  in  other  information  that  had 
not  yet  been  includ  ‘d  in  their  planning.  After  we  did 
this  several  times,  opened  it  and  then  crystallized  it 
again  and  again,  we  then  put  on  the  show.  But  we 
changed  our  plan  only  30  minutes  before  we  started.  It 
just  about  scared  the  wits  out  of  those  in  power  because 
they  said  that  we  still  hadn’t  figured  out  two  or  three 
more  details  of  the  plan.  They  asked  how  did  we  dare 
to.  open  it  up  and  change  it  at  this  late  stage  when  we 
hadn’t  even  quite  finished  making  the  plan.  But  we 
never  did  give  the  participants  an  agenda;  they  never 
asked  for  one;  and  we  were  keyed  up  creating  the  new 
plan  as  the  show  kept  rolling.  We  didn’t  schedule  our 
cofleebreaks  or  anything  else  but  let  them  happen  when 
it  was  timely  to  do  so,  we  just  kept  everything  rolling 
and  the  people  enjoyed  it  very  much.  Admittedly,  a 
couple  of  people  who  were  helping  us  run  it  were  under 
some  strain;  they  came  up  to  us,  quite  upset,  because 
we  weren’t  following  specific  enough  plans.  Later  one 
friendly  critic  said,  “Gee,  you  look  so.  unplanned  and 
relaxed.  Tell  us  how  long  did  you  have  to  plan  to  get 
that  way?” 

Let  me  go  a  little  into  the  problems  of  selection.  As 
indicated  earlier,  we  would  probably  use  a  biographical 
inventory  as  our  best  single  basis  for  selecting  those 
high  on  either  planning  or  long-range  forecasting.  By 
learning  to  read  appropriate  parts  of  the  life  program 
of  a  person,  we  already  know  how  to  forecast  the  future 
creativity  of  a  person.  If  in  the  past  a  person  has  largely 
kept  his  creating  processes  dormant  and  has  not  used 
them,  then  he  is  not  a  good  prospect  next  year  to  be  a 
creative  person,  whereas  another  person  who  has  had 
his  creative  processes  working  and  has  been  in  climates 
and  situational  experiences  where  year  after  year  these 


processes  are  functioning,  then  we  can  forecast  that 
during  the  next  5  years  he  is  going  to  be  a  good  pros¬ 
pect  to  be  creative,  ^e  should  likewise  work  very  hard 
to  build  an  inventory  for  measuring  planning  talents 
(and  another  one  for  forecasting  talents)  along  this 
same  kind  of  line.  First,  we  would  generate  a  thousand 
multiple  choice  items  and  try  to  find  150  of  the  best 
ones  that  we  could  find  that  display  a  life  history  lead¬ 
ing  to  a  high  degree  of  effective  planning  talents.  We 
could  also  select  among  those  experienced  in  planning 
to  try  to  get  some  measure  of  the  batting  average  of 
those  experienced  in  each  of  these  activities.  Then  we 
would  find  the  biographical  items  that  select  the  better 
planners  from  the  rest  and  then  repeat  the  procedure 
to  find  the  better  forecasters  from  the  rest. 

Let  me  point  out  some  highlights  in  the  work  of 
Guilford  and  his  staff  (1964-65)  at  the  University  of 
Southern  California  in  a  study  of  planning  abilities. 
They  did  a  first  general  study  of  basic  planning  talents, 
so  let  me  give  you  some  titles  of  the  sets  of  character¬ 
istics  they  were  studying.  One  was  orientation,  includ¬ 
ing  sensitivity  to  order,  discovery  of  conceptual  rela¬ 
tions,  and  recognition  of  variables.  A  second  was  elab¬ 
oration  including  specifying  details,  producing  ideas 
and  alternate  methods,  and  symbolizing  ideas.  The 
third  was  ordering,  arranging,  and  integrating  includ¬ 
ing  both  temporal  and  hierarchial  ordering.  The  fourth 
was  originality  and  ingenuity  in  new  methods  and  ap¬ 
plications  and  adaptive  flexibility.  The  fifth  included 
an  ability  to  put  all  this  together  into  a  prediction 
where  you  have  to  use  such  talents  as  visualizing  things 
you  can’t  actually  see,  or  having  foresight  through 
ideas  and  through  things  you  can  perceive^  and  extra¬ 
polating  from  the  knowns  to  the  knowns.  And  finally, 
things  like  judging  and  evaluating  (a  high  degree  of 
which  would  be  wisdom),  including  seeing  shortcom¬ 
ings  in  plans,  and  judging  the  importance  of  variables. 
In  summary,  there  are  a  lot  of  talents  and  processes 
and  activities  in  planning,  as  best  understood  so  far, 
and  the  beginnings  of  measurement  exist  in  these  above 
areas. 

« 

One  consulting  group  is  working  vigorously  in  the 
field  of  planning.  They  feel  that  if  they  would  put  to¬ 
gether  everything  now  known  about  planning,  it  would 
make  a  major  book.  Through  such  a  book  they  feel 
they  could  get  planning  training  into  all  management 
departments  in  schools  of  business  across  the  Nation  in 
about  3  years  elapsed  time.  But  is  someone  doesn’t  ac¬ 
tively  take  such  steps,  they  feel  it  will  take  about  30 
years  to  gel  what  is  now  known  into  the  system  because 
of  the  slowness  of  the  bridging  process  now  going  on. 
We  have  an  organization  with  a  group  of  students  and 


former  students  who  are  ready  not  only  to  tvork  in 
creativity  and  to  handle  the  snowballing  of  activity 
there,  but  also  to  move  into  these  other  areas  of  high 
level  talents  as  they  really  occur  in  the  world  of  work. 

Let  us  think  briefly  about  the  combination  "of"p]an-'  “ 
ning,  creativity,  and  flexibility  in  planning.  I  have  indi¬ 
cated  certain  problems  of  communication  which  I  think 
are  very  important:  The  ability  to  listen  to  new  infor¬ 
mation  and  new  ideas  from  others,  and  also  to  listen 
to  yourself,  to  find  value  and  use  for  that  which  you 
already  have  within  yourself,  so  it  is  not  only  listening 
and  receiving  from  outside  but  also  receiving  ^rom 
inside.  Some  of  the  great  creative  people  are  the  ones 
who  so  fully  use  the  stuff,  the  inner  resources  already 
within  themselves.  Perhaps  the  same  thing  is  true  of 
great  planners  and  of  great  forecasters.  They  must 
know  how  to  use  their  own  full  background  of  experi¬ 
ence  as  well  as  the  immediate  surrounding  information 
that  is  available  to  them. 

Let  me  tell  you  a  sad  finding  in  creativity,  which  may  ' 
also  be  true  in  planning.  If  you  try  in  turn  to  stand  in 
the  shoes  of  the  two  types  of  people  who  are  involved, 
the  person  with  the  idea  and  the  other  person,  often 
with  the  higher  statu^  you  may  get  a  better  feel  of  the 
phenomenon.  The  finding  goes  like  this:  the  more 
highly  creative  an  idea  a  person  has  at  any  time  in  his 
life,  the  more  likely  he  is  to  be  in  trouble.  Both  he  and 
his  creative  idea  will  probably  be  in  trouble  with  other 
people  and  vrith  the  system  or  establishment  and  its 
features. 

A  new  creative  idea  is  likely  to  run  into  an  old  plan 
and  usually  someone  else’s  plan  who  probably  is  higher 
in  the  organization.  Likewise,  a  new  idea  will  run  into 
an  old  system  built  by  someone  else  who  usually  is  in 
greater  power  than  the  suggester  of  the  new  idea. 

Maybe  along  the  same  vein,  the  more  unexpected 
though  high-quality  plan  a  person  has  at  any  time  in 
his  life  (if  he  isn’t  sitting  as  vice  president  in  charge  of 
planning)  and  the  more  unex^cted  it  is  and  the  less  it 
resembles  the  present  scheme  of  things,  and  the  more 
likely  he  and  his  plan  are' to  be  in  trouble  with  other 
people  and  with  the  established  organization.  The  peo¬ 
ple  and  the  system  will  not  separate  him  from  his  plan 
but  will  tend  to  clobber  them  both  together. 

Results  and  observations  like  the  above  ones  strongly 
suggest  that  organizations  are  often  at  their  best  on 
matters  that  are  quite  routine  and  of  minor  importance. 
In  sharp  contrast,  however,  organizations  are  often  at 
their  absolute  worst  on  matters  of  highest  importance, 
such  as  (1)  when  a  quick-reaction  capability  is  sorely 
needed  in  responding  to  unexpected  but  marvelous 
opportunities;  (2)  when  the  organizations  are  asked  to 


respond  properly  to  creative  individuals  and  to  the 
creative  processes  and  creative  ideas  from  within 
themselves;  and  (3)  when  they  should  become  flexible 
and  adjust  themselves  away  from  their  old  plans  toward 
new -plans  and  in  directions  indicated  by  brilliant  new 
ideas. 

I  have  given  you  a  little  feel  of  flexibility  in  planning. 
Some  organizations  have  had  an  opportunity  knock  on 
the  door  and  their  individuals  could  hear  that  oppor¬ 
tunity  and  knew  it  was  there,  but  the  organization  as  a 
whole  officially  would  not  listen  because,  for  example, 
they  were  already  planned  and  programed  for  the  next 
2  years.  Instead,  they  could  start  thinking  about  putting 
that  opportunity  in  the  program  3  or  4  years  from  now, 
but  they  could  not  think  of  adjusting  the  present  pro¬ 
gram  right  now  in  terms  of  that  opportunity.  They  had 
what  I  call  a  slow  reaction  capability  of  at  least  2  or  3 
years  lag  time.  I  am  sure  some  of  you  have  encountered 
organizations  that  say,  “Let  the  present  plan  run  its 
course  before  we  consider  new  ideas.” 

There  is  at  least  one  instance  when  for  450  years  an 
organization  had  shown  an  Incapability  of  listening 
before  it  finally  became  capable  of  listening — but  that 
lag  time  is  better  than  infinity.  Those  currently  in  the 
organization  would  not  live  long  enough  to  see  their 
ideas  or  plans  accepted  with  a  450-year  lag  time  for  the 
organization  to  listen  to  what  might  be  valuable  for  it. 
You  will  find  individual  differences  in  persons  and  in 
organizations  on  listening  ability  and  readiness  and 
receptiveness  to  new  ideas.  Two  of  the  slogans  you  rnay 
have  heard  are:  “Don’t  confuse  me  with  further  infor¬ 
mation.  My  plan  is  already  made  up.”  The  other  one  is, 
“If  you  don’t  understand  it,  oppose  it.”  The  new  infor¬ 
mation  or  new  ideas  may  be  difficult  to  imderstand 
because  of  fixed  ideas  about  the  old  plans  and  they  may 
mean  a  new  plan  or  replanning  would  be  required 
which  might  be  resisted  as  not  being  worth  the  trouble. 

We  have  wondered  how  to  get  versatility  and  flexi¬ 
bility  and  a  variety  of.plans  and  quick-reaction  capa¬ 
bility  into  organizations  so  that  instead  of  having  only 
one  plan,  the  organization  may  have  multiple  plans.  In 
the  military,  the  cardinal  sin  is  to  have  one  and  only 
one  plan,  without  an  alternate  plan.  How  can  you  de¬ 
velop  flexible  plans  so  that  they  will  still  work  in  terms 
of  the  whole  organization?  This  is  a  very  challenging 
problem. 

And  how  do  organizations  deal  with  new  ideas  and 
new  plans  that  emerge  from  below  in  terms  of  uncrys- 
tallizing  plans  or  of  having  flexible  or  tentative  plans? 
How  can  one  get  those  in  positions  of  control  to  be 
more  flexible  concerning  plans?  They  may  all  need  a 
high-level  profile  across  talents,  but  we  often  have  to 
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work  with  the  people  who  are  now  in  the  positions,  no 
matter  what  their  profile  of  talents  looks  like. 

Let  me  give  an  example,  which  is  both  seasonal  and 
appropriate.  In  professional  football  they  really  have 
many,  many  plans  for  their  entire  organization  (or 
team)  to  follow.  They  have  many,  msny  plans  and 
every  player  is  well  versed  in  each  and  every  one  of 
these  plans  and  knows  how  to  carry  each  one  out  lo 
form  a  well-coordinated  team.  Instead  of  a  single  plan, 
they  have  many,  many  plans,  and  they  also  have  quar¬ 
terbacks  who  have  a  sequence  or  strategy  of  plans. 
They  have  a  game  plan  which  is  a  whole  series  or  pat¬ 
tern  of  plays,  including  thi  igs  to  carry  out  which  are 
supposed  to  open  up  the  other  team.  Then  the  other 
thing  1  like  about  the  smooth  functioning  professional, 
football  team  is  that  they  have  quick  reaction  capa¬ 
bility.  They  arfen’t  too  tightly  planned,  though,  and  if 
the  "day  is  broken  up,  they  still  somehow  scramble  and 
try  to  make  a  good  play  out  of  it.  Also,  when  they  are 
in  a  huddle,  the  quarterback  is  open  for  input  from 
other  players.  If  one  of  the  players  says,  “I  see  some¬ 
thing  unusual,”  they  may  even  create  a  new  play  in  the 
huddle.  And  at  the  last  moment  when  the  quarterback 
comes  up  to  the  line  of  scrimmage,  he  is  still  open  for 
input  from  the  other  team’s  defensive  arrangement. 
And  the  quarterback  can  and  does  often  change  a  play 
at  the  line  of  scrimmage.  So  you  see  they  are  well 
planned,  but  they  also  have  great  flexibility  in  their 
planning  so  that  they  can  try  to  avoid  getting  into 
trouble.  Those  who  don’t  have  this  flexibility  in  either 
offense  or  defense  are  vulnerable  to  being  put  into 
trouble  by  a  more  flexible  and  creative  though  also  well 
organized  and  planned  opponent. 

This  brings  up  the  point  that  the  more  we  can  learn 
how  to  release  the  creative  potentials  in  people  so  that 
there  is  more  creativity  in  action,  the  more  exciting  and 
unpredictable  the  future  can  be.  Creativity  when  effec- 
tivtly  funtioning  can  change  capabilities  and  feasibili¬ 
ties.  ijo  creativity  can  thereby  upset  the  long-range 
forecasts  and  may  also  cal!  for  a  change  in  the  plans, 
if  only  to  keep  in  adjustment  with  the  creative  strides. 


I  would  like  to  close  with  a  somewhat  subtle  story, 
which  I  hope  you  will  enjoy.  A  young  fellow  had  done 
well  in  the  academic  world  in  meteorology  and  felt  he 
had  a  promising  future  in  weather  forecasting.  So  he 
got  his  first  assignment  in  a  place  like  New  Mexico 
where  one  can  nearly  always  forecast  that  the  sun  is 
going  to  shine.  He  did  quite  well  in  his  first  job  and 
was  really  very  happy  in  his  work  and  wan  moving 
along  successfully.  The  Weather  Bureau  also  liked  his 
work,  so  when  they  had  an  opening  in  Nev  England, 
they  decided  to  select  him  to  fill  that  spot.  They  believed 
that  if  they  could  gel  the  right  man  in  the  right  place, 
their  problems  would  be  largely  over — but  if  they  got 
the  wrong  man  in  the  right  place,  their  problems  would 
just  begin.  So  they  selected  him  and  sent  him  to  New 
England.  When  he  got  up  there,  he  found  that  he  had 
a  different  world  in  whieh  to  work.  The  weather  was 
much  more  complex  and  changeable  and  unpredictable, 
so  that  “if  you  didn’t  like  the  weather,  you  .should  just 
v/ail  a  minute.”  He  worked  hard  in  this  new  setting  and 
made  his  forecasts,  but  after  a  while  he  didn’t  have 
anywhere  near  the  halting  average  he  had  previously 
had  in  this  New  Mexico  sunshine  State. 

He  struggled  with  this  problem  and  tried  to  perfect 
his  techniques,  but  could  only  improve  them  a  very 
slight  amouiit  under  this  new  and  difficult,  many-vari- 
abled  situation.  Aftsr  a  while,  some  of  his  neighbors 
started  kidding  him  and  other  people  stopped  him  on 
the  street  and  would  tease  him,  and  then  the  newspaper 
picked  it  up  and  slc.rted  making  alternate  forecasts  from 
someone’s  bunion,  and  so  on.  Gradually,  all  of  this 
started  to  trouble  him  greatly.  Finally,  he  wasn’t  thick 
skinned  enough  to  handle  it  all,  so  it  started  to  really 
get  to  him.  He  wrote  to  the  Weather  Bureau  in  Wash¬ 
ington,  D;C.,  and  said,  “I  would  like  to  transfer  else¬ 
where.”  The  Weather  Bureau,  after  having  felt  that  they 
had  the  right  man  in  the  right  place  and  could  stop 
worrying  for  a  long  time,  were  caught  by  surprise. 
They  immediately  sent  a  telegram  to  him  stating,  “We 
don’t  understand.  Please  explain.”  He  replied  with  the 
simple  answer  that  “The  weather  doesn’t  agree  with 
me.  * 
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Mr.  Cetron.  There  is  an  old  saying,  “in  the  country 
of  the  blind,  the  one-eyed  man  is  king.”  I  don’t  really 
believe  it.  I  think  they’ll  poke  bis  eye  out.  Hiis  is  a 
problem  with  any  man  who  is  ahead  of  his  time,  espe¬ 
cially  in  large  organizations.  If  we  take  one  of  these 
creative  people,  that  we  trained  from  kindergarten,  he’ll 
get  to  the  manager,  who  will  say  “Fine,”  pat  him  on  the 
head  and  forget  it.  We  try  to  fo  x  everybody  into  the 
same  mold  and  it’s  particularly  true  in  the  military. 

Dr.  Taylor.  I  recall  a  statement  which  says,  “Divide 
and  conquer.”  As  long  as  we  keep  our  research  going; 
where  we  deal  with  one  position  at  a  time,  we  can  get : 
cooperation.  If  we  have  to  deal  with  the  whole  organi¬ 
zation,  we  will  have  problems.  You  might  call  it  in¬ 
formal  versj’s  fonnal  organizations,  and  we  are  trying 
to  work  through  the  informal.  This  is  very  central  in 
my  thinking.  Our  next  book  is  “Climate  for  Creativity,” 
and  my  chapter  is:  “Can  organizations  be  creative, 
too?”  If  individuals  in  organizations  can,  can  organi¬ 
zations  as  a  whole?  There  is  a  whole  series  of  charac¬ 
teristics  you  seek  in  organizations.  Can  we  get  organi¬ 
zations  functioning  at  this  high  level?  That’s  what  I’m 
talking  about;  The  whole  team  having  quick  reaction 
capability,  being  able  to  adjust  as  a  team  to  the  latest 
information.  In  the  competitive  game  of  pro  football, 
we  have  reached  that  and  in  the  competitive  game  of 
organization,  it  must  be  functioning  this  way.  One 
organization  with  which  I  met  said,  “The  things  we  are 
doing  this  year,  we  were  not  doing  last  year.”  So  they 
changed  almost  a  hundred  percent.  Some  organizations 
do  have  very  fast  reaction  capabilities  in  planning  and 
creating  They  are  creating  the  world  of  the  future.  It  is 
others  that  aren’t  adjusting  who  will  be  on  high  and 
dry  land. 

As  far  as  the,  “can’t  doers”  and  “it’s  impossible,”  in 
our  Peace  Corps  approach  we  built  situational  tests 
where  this  could  be  a  dodge  and  if  a  person  took  the 
dodge  out  of  the  .<4ituationai  test,  the  sooner  he  flunked 
out  of  the  Peace  Corps,  as  far  a»  our  proposed  testing 
was  concerned.  So  we’ve  played  this  game.  There  ate 
many  people  sitting  in  key  positions  who  know  the 
reason  why  it  can’t  be  done  and  are  making  no  contri¬ 
bution  except  to  kill  things.  Other  people  who  have  the 
reputation  as  can-doers — no  matter  what  you  take  to 


them — ^will  .find  a  way  to  help  you.  Like  the  mayor  of 
Fairbanks  s9i4>  “We  are  a  can-do  community  and  we’re 
going  to  find  i^ays  to  plan  and  carry  this  out.” 

Dr.  LiNS^pi^EJ  I  agree  with  Mr.  Cetron  only  there’s 
one  thing  yOtf  pbntioned,  tentative  planning.  It  is  very 
useful  and  ||  ((llink  it  helps  in  a  great  number  of  situa¬ 
tions.  The  ^himiht  of  the  planner  is  that  he  doesn’t 
consider  or  p^pose  to  the  decisionmaker  how  to  get 
from  here  to  in  small  steps.  He  goes  there  in  one 
step.  This  nj^i^^.li  great  risk  on  the  part  of  the  decision¬ 
maker.  It  helps' very  often,  and  I  found  more  recep¬ 
tivity,  ihdicate  not  just  a  plan  but  also  you 

might  say  the'  off  plans,  where,  if  you  are  wrong,  he 
hasn’t  committed  himself  too  much.  In  other  words, 
there  are  additional  points  on  the  way.  You  don’t  give 
one  single  best  way  to  get  from  here  to  there  but  a 
good  way,  not  as  good  as  the  best  way,  which  permits 
alternatives.  Ihis  is  nothing  more  than  dynamic  pro¬ 
graming  in  which  the  decision  is  made  on  the  informa¬ 
tion  available  then.  You  will  find  he  is  much  less 
opposed  anil  is  willing  to  take  this  risk  because  he  sees 
he  doesn’t  have  to  commit  himself  for  15  years,  and  a 
tremendous  amount  of  money,  which  is  obviously  im¬ 
portant.  This  tentative  planning  is  very  good. 

Dr.  Taylor.  I  have  two  comments.  First,  I  have  been 
working  with  a  composer  and  he  says,  “A  good  com¬ 
poser  composes  just  the  right  amount;  not  too  much, 
not  too  little.  Too  little,  he  ddesri’t  give  enough  of  a 
pattern  and,  too  much,  he  doesn’t  give  the  person  who 
is  going  to  carry  out  the  plan  a  chance  for  his  own 
individual  imagination  to  make  it  really  and  truly 
artistic,  to  make  a  creative  performance.” 

The  second,  we  had  a  dissertation  in  the  field  of 
mathematics,  in  creativity.  It’s  a  fascinating  one.  The 
person  himself  chose  to  be  two  different  kind  of  teach¬ 
ers:  the  typical  kind  of  teacher  in  the  morning,  the 
learners’  teacher  and  select  story  problems  in  the  area 
of  work.  In  the  afternoon,  he  was  going  to  be  a  teacher 
where  the  studev.ts  were  thinkers.  What  happened?  In 
the  morning  the  students  learned  the  typical  way.  The 
teacher  told  them  the  solution  to  the  problem,  the  one 
and  only  way  to  get  to  the  solution.  There  was  one  and 
only  one  route  and  that  was  all  they  >yere  supposed  to 
work — the  teacher’s  way,  the  textbook  'way,  and  so  on. 


26 


In  the  afternoon  the  teacher  had  to  play  a  different  role. 
The  students  generated  all  the  routes  they  could  go  to 
Rome  and  learned  each  other’s  routes  and  so  on.  In  the 
end  what  happened  was:  the  students  in  the  afternoon 
didn’t  work  on  nearly  as  many  problems  as  the  students 
in  the  morning.  But  when  tested  on  an  exam  guess  who 
could  work  the  story  problems  better?  The  students  in, 
the  afternoon,  who  enjoyed  working  story  problems, 
who  had  to  listen  and  learn  about  these  different  solu¬ 
tions.  The  teacher  had  to  listen  to  the  students  to  see  if 
their  solutions  were  feasible  and  to  think  with  them, 
where  before  he  had  pulled  out  the  textbook  answer 
and  said,  “Yours  doesn’t  fit  this  so  it’s  wrong,”  where 
it  might  have  been  an  alternate  route.  When  the  stu¬ 
dents  were  asked:  where  did  we  spend  too  much  time 
per  problem,  which  group  of  students  answered  this? 
The  ones  in  the  morning  said  they  spent  too  much  time 
on  a  problem.  The  afternoon  students  actually  spent 
more  time  on  a  problem  than  the  morning  students  but 
did  not  make  this  complaint  because  they  enjoyed 
studying  the  routes  to  Rome  and  were  interested  in 
finding  a  solution  more  than  the  morning  students.  The 
afternoon  students  didn’t  care  how  they  got  the  solu¬ 
tion.  just  so  they  got  one;  where  the  morning  students 
were  more  interested  in  the  routes  to  Rome  than  the 
final  solution.  We  are  products  of  this  kind  of  training. 
If  we  were  trained  in  that  afternoon  class,  we  would  all 
probably  have  multiple  routes  in  mind,  if  we  got  this 
far,  maybe  we  could  go  over  lu  another  route  and  so  on. 
I  think  we’d  have  a  much  greater  flexibility  in  our¬ 
selves  if  we  would  have  been  trained  this  way. 

Dr.  Dalkey.  Let  me  elaborate.  Where  an  individual 
is  most  likely  to  have  highly  creative  ideas,  he  is  also 
most  likely  to  be  in  trouble.  Wliat  kind  of  trouble  were 
you  referring  to  and  were  you  suggesting  that  being  in 
trouble  is  part  of  the  environment  that  is  favorable  to 
creativity  or  the  reverse? 

Dr.  Taylor.  In  my  latest  writing,  it  says  it  is  hard 
for  this  poor  guy  to  read  the  symbols — the  signals 
coming  hack  to  him.  Beiiig  highly  “clobbered”  may  be 
the  highest  form  of  recognition  some  people  can  get. 
There  are  about  eight  or  70  research  studies  that  add 
up  to  this  picture  and  one  says,  if  you  get  youngsters 
together,  quite  small  youngsters  comparatively,  and 
have  them  think  up  new  ideas  in  a  group,  and  one  is 
starting  to  outstrip  the  others  too  far,  the  group  will 
tenci  to  say — these  are  group  pressures — we’ve  got  to 
get  organized;  we  aren’t  organized.  One  of  the  kinds 
of  thing  they  will  do,  will  say :  we’ve  got  to  write  some 
rules.  When  they  write  the  rules,  they  don’t  write  them 
to  facilitate  progress  hut  to  constrain  people  and  the 
one  they  are  trying  to  constrain  is  this  fellow  who  is 


thinking  too  fast  and  far  and,  I  think,  unexpectedly  for 
them.  Many  times  rules  are  written  into  organizations 
by  people  who  are  more  interested  in  controlling 
progress,  maybe  trying  to  control  creative  people.  The 
first  thing  plan  suggestors  do  is  say,  “We’ve  got  to  have 
various  members  of  the  group,”  so  they  try  to  organize 
again  and  ge*  a  chairman,  secretary  and  a  recorder  and 
things  of  this  type.  Guess  what  they  do  with  the  fellow 
who  is  outstripping  them?  Make  him  the  recorder  so 
this  takes  care  of  him  since  he  is  busy  writing  their 
ideas  down  it  gets  him  out  of  the  ball  game.  They’ve 
organized  him  in,  unless  he  writes  a  minority  report 
and  doesn’t  record  it.  The  other  tendency  is  if  they 
can’t  organize  him  in,  they  organize  him  out,  and  use 
group  pressure  which  I  hinted  at  earlier.  The  highly 
creative  people  who  want  to  be  leaders  also  want  to  be 
surrounded  by  a  group.  The  whole  world  may  follow 
them  later  when  the  world  finally  makes  enough  prog¬ 
ress  to  get  up  that  way,  so,  if  you  bring  in  a  new  idea 
that  barely  inches  ahead  and  no  one  has  to  do  too 
much  to  adjust  to  it,  it  is  easily  communicated  because 
there  is  just  so  much  expected  and  it’s  just  a  wee  bit 
new.  But,  if  you  are  bringing  a  new  idea  that  is  highly 
unexpected  and  is  going  to  cause  trouble  and  work, 
many  people  in  key  positions  won’t  go  that  route.  You 
go  to  all  this  work  because  you  have  an  idea  and  they 
are  in  a  lower  status  and  these  are  the  kind  of  prob¬ 
lems  one  runs  into.  He  is  all  alone,  in  the  first  place,  he 
and  his  idea,  against  the  world.  Whether  he  can  get 
other  people  to  join  this  idea  is  one  of  the  problems 
and  it  is  sad  but  the  higher  the  percentage  of  people 
against  you  may  be  an  indication  of  the  degree  of 
unexpectedness  and  potentially  the  degree  of  creative- 
ness  of  new  ideas. 

Dr.  Dalkey.  There  is  a  big  difference  between 
having  a  talent  and  employing  it.  I  was  wondering  if 
you  had  reflected  at  all  on  the  long  series  of  studies 
which  are  gradually  seeping  into  public  view  with 
regard  to  the  relationship  between  child  order  and 
eminence?  The  studies,  for  example,  demonstrated  in 
“Who’s  Who”,  the  probability  that  the  first  child  is 
successful  much  higher  than  it  is  for  total  population, 
and  that  this  is  a  way  of  increasing  the  function  of  the 
order  of  the  childbirth  pattern. 

Dr.  Taylor.  Incidentally,  my  latest  paper  i.  called 
“Leadership  toward  Creative  Organization  and  Func¬ 
tioning.”  It  deals  with  trying  to  get  a  leader  to  function 
so  it  is  the  creative  function  of  a  whole  organization. 
We  have  done  more  biographical  studies  of  scientists 
and  creative  people  by  far,  and  have  gone  through  the 
computer  about  a  hundred  times  on  a  hundred  different 
criteria  of  performance  with  20  or  more  samples,  15  or 
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more  samples.  I  don’t  think  we  have  found  this  as 
strong  as  you  indicate  about  order  and  eminence.  One 
of  our  problems  is  we  cannot  publish  this  or  we  spoil 
the  scoring  key.  We  have  people  write  themselves  up  in 
the  form  of  an  application  blank.  Then  we  go  back  in 
life  history  to  see  what  kind  of  experiences  lead  to 
being  a  creative  person,  but  I  don’t  think  we  found  it 
as  strong.  In  fact,  I  can  say  with  confidence,  no  single 
item  by  itself  will  give  you  very  strong  predictions.  We 
have  to  use  150  or  more  collectively  to  give  a  good 
solid  report. 

Mr.  Irwin.  I  was  interested  in  the  first  point  Dr. 
Dalkey  made — the  question  of  whether  the  creative 
person  is  beaten  down  by  the  group.  All  of  Dr.  Taylor’s 
responses  were  very  much  predicated,  it  seems  to  me, 
on  our  own  society.  What  happens  in  American  schools 
today  or  yesterday  or  even  20  years  ago,  when  a  person 
or  a  child  shows  creativity?  I  personally  want  to  speak 
vigorously  against  the  idea  that  it  is  a  good  th  ,g  to 
have  this  creative  person  jumped  out  of  the  mass  in 
response  to  the  pressure.  This  talent  is  so  great  that  he 
achieves  his  creativity  anyway.  It’s  true;  it  happens. 
But  I  want  to  refer,  as  my  authority,  to  Eric  Hoffer 
whose  ideas  affect  me  a  great  deal  and  I  offer  it  for 
discussion :  Hoffer’s  thesis  is  that  everybody  is  creative 
and  if  you  and  society  encourages  the  creativity,  the 
mass  creativity  that  comes  from  this  is  enormously 
greater  and  he  cites  the  type  of  painting  that  came 
from  Florence,  out  of  a  very  small  numerical  group  of 
people,  who  produced  a  tremendous  amount.  He  cites 
the  Elizabethan  period  with  their  tremendous  flowering 
in  poetry  and  music  and  I  am  very  much  opposed  to  the 
idea  that  it  is  good  for  creativity  to  make  him  jump 
out  as  a  group. 

Dr.  Taylor.  You  are  saying  that  some  people  say 
creativity  will  rear  its  head  no  matter  what  and  will 
fight  its  way  through.  This  past  summer  we  had  Dr. 
Toynbee  on  our  campus,  to  give  two  lectures.  We  are 
now  in  the  process  of  getting  out  19,000  copies  of  the 
two  lectures  in  creativity.  He  defined  creativity  as  that 
talent  which  when  turned  into  activity  in  enough 
people  in  society,  will  create  history  in  any  field  of 
human  activity  they  are  engaged  in,  and  asks  that  more 
Americans  engage  in  this  because  they  are  at  present 
neglecting  its  future  history.  He  says  creative  talent  is 
the  ultimate  capital  asset  of  mankind  of  any  society  and 
it  is  a  matter  of  life  or  death  for  any  society.  He  said  if 
America  is  to  have  a  manifest  interest  to  spark  the  en¬ 
tire  world,  then  we  must  use  all  the  creative  potential 
that  we  have.  All  the  studies  of  creativity  indicate  wide 
individual  differences  in  the  amount  of  potential  or 
actual  creative  talent  that  we  would  have.  It  doesn’t 


say  anyone  is  zero.  Most  of  the  contributions  and 
studies  say  the  responsibility  is  on  other  people’s 
shoulders,  a  whole  flock  of  other  people’s  shoulders.  So 
if  you  can  get  the  whole  shoulders  raised,  you  don’t 
have  as  far  a  hike  to  go,  to  go  higher.  So  our  attempts 
to  get  creativity  in  the  classroom  are  consistent  with 
what  you  are  saying.  Some  will  rear  their  head  no 
matter  what  and  others  won’t.  We  want  this  to  rise  to 
higher  levels  so  we  want  other  talents  to  go  with  it.  In 
American  schools  evidence  shows  there  is  a  fourth 
grade  slump  in  thinking  abilities  but  there  isn’t  a  slump 
in  height  and  weight  and  so  on.  When  this  was  tested 
in  other  cultures,'  a  slump  was  not  indicated,  however, 
Americans  go  around  the  world  and  try  to  be  of  help 
in  other  cultures.  One  place  in  Samoa,  they  tried  to  help 
the  fourth  grade.  What  did  the  Americans  help  the 
fourth  grade  to  get?  A  slump  in  their  thinking  abilities. 

Mr.  Irwin.  This  is  one  proof  of  the  point  I  was 
making.  I  have  a  boy,  5,  in  kindergarten.  Everybody 
draws  a  picture  and  they  are  all  enormously  creative, 
very  original.  There  are  no  restrictions  and  all  their 
paintings  are  exciting.  Then  they  run  into  the  attitude 
of  society.  He  is  just  emerging  and  hardly  knows  he  is 
a  person.  As  soon  as  he  does  then  he  finds  there  is  this 
restriction.  It  has  got  to  be  removed.  There  will  be  an 
enormous  increase. 

Dr.  Taylor.  Again  this  is  my  point.  Norman  said 
there  is  a  strange  thing  in  history,  a  fraction  of  society 
becomes  effective  in  terms  of  creativeness  and  rise  to 
help  make  that  part  of  the  world  rise  to  a  place  in 
history.  The  interesting  thing  is,  in  the  course  of  his¬ 
tory,  that  same  society  starts  turning  its  back  on  the 
very  talents  that  had  it  rise  to  its  place  in  history.  They 
even  try  to  neutralize  these  talents  when  they  start  to 
emerge.  That’s  the  alkali  on  the  acid  when  it  starts  to 
sparkle,  so  he  is  saying  this  is  what  the  natural  tend¬ 
ency  of  history  is:  Why  some  people  have  their  names 
in  history — and  you  are  saying  that  America  is  show¬ 
ing  too  much  of  this  and  cannot  reverse  itself.  We 
have  had  our  day  in  history. 

Dr.  Manz.  I  think  I  can  reconcile  between  the  tv^o 
points  if  you  modify  your  statement  a  little,  by  saying: 
I  know  many  great  mathematicians  and  one  in  par¬ 
ticular,  Felix  Linev.all,  has  said,  on  several  occasions 
that  mathematics  could  be  understandable  and  could 
be  an  intellectual  pleasure  to  everyone  but  it  isn’t,  be¬ 
cause  so  many  teachers  are  failures.  They  do  not  have 
the  ability  to  bring  this  pleasure  to  the  student.  I  realize 
this  in  my  own  son.  He  hates  mathematics  and  physics 
and  I  cannot  help  it — he  is  appalled  by  it — because  of 
the  miserable  way  mathematics  and  physics  are  taught 
in  high  school.  An  even  worse  situation  is  facing  us 


today.  There  have  been  several  articles  written  that  the 
present  teaching  of  mathematics  at  universities  indulges 
in  formalism,  ugly  emptiness,  and  therefore  does  not 
really  appeal  to  any  creative  mind.  We  have  an  interest¬ 
ing  phenomeon,  many  of  our  most  intelligent  people 
turn  their  backs  on  science  and  go  into  other  areas.  I 
do  not  mean  to  degrade,  they  are  important,  but  we 
do  have  a  degradation  of  our  scientific  education. 
Creativity — when  you  make  the  statement  “everybody 
has  creativity,”  this  I  would  modify.  I  vould  say 
creativity  is  something  you  either  have  or  you  do  not 
have.  Therefore,  I  am  not  sure  about  that  statement, 
but  if  you  say  “talent,”  I  absolutely  agree.  Everybody 
has  talent  unless  he  is  a  complete  idiot  and  if  it  so 
happens  that  a  student  dislikes  a  certain  field  at  school 
or  university  then  it  is  the  fault  of  the  teacher.  As  I 
said  before,  mathematics  is  something  that  has  so  much 
beauty  and  intellectual  appeal  that  it  must  appeal  to 
everybody  and  if  it  isn’t  so,  then  something  is  wrong 
with  the  system  of  education. 

Dr.  Taylor.  My  previous  statement  was  a  search  for 
a  creative  climate.  Every  study  we  made  indicates  what 
we  have  in  the  way  of  a  climate  for  creativity  is  not 
ideal.  It  is  not  anywhere  near.  Changes  are  necessary. 
In  our  proposal  of  education,  we  will  be  developing 
the  talents  of  the  various  students  as  they  go  through 
acquiring  subject  matters,  we  are  trying  to  have  these 
students  experience  different  talent  processes  while  they 
are  working  with  the  subject  matter.  They  are  using 
this  infiltration  test  during  the  subject  matter,  trying 
to  get  the  youth  to  use  these  new  talent  processes.  The 
students  are  finding  the  schools  much  more  interesting 
since  they  get  a  chance  tc  use  this  variety  of  talents. 
The  students  are  ready— it’s  the  system  that  isn’s  ready. 
The  students  are  ready  for  the  opportunity. 

Dr.  Slafkosky.  I  think  in  a  sense  you  were  mis¬ 
understood.  I  didn’t  think  you  were  saying  that  it’s  a 
matter  of  fact  creativity  had  to  spring  from  a  time  or 
a  period  of  held-down  influence.  It  often  has  in  the  past 
and  it  certainly  is  frequently  the  case  now.  An  answer 
to  the  question  about  the  function  of  American  culture 
is,  If  you  look  at  the  scientific  areas  or  philosophic 
areas  throughout  the  centuries,  Socrates  wasn’t  recog¬ 
nized — he  had  to  take  the  poison.  Galileo  wasn’t. 
Neither  was  Copernicus.  Their  contemporaries  ridi¬ 
culed  them.  This  is  the  function  not  only  of  our  society 
but  the  whole  western  tradition.  Granted  we  want  to 
eliminate  or  ameliorate  it,  but  I  do  think  the  idea  you 
were  propounding  was  mainly:  Let’s  make  it  easy  for 
the  creativity,  first,  to  be  recognized  and  for  it  to 
flourish.  You  didn’t  necessarily  say  it  had  to  be  done 
in  adversity.  Do  I  understand  you  properly? 


Mr.  Irwin.  I  didn’t  take  this  as  being  directed  at  me. 
I  thought  he  was  talking  to  people  who  had  expressed 
the  idea  in  the  field  of  creativity  because  I  did  not 
express  the  idea  he  was  commenting  on.  I  dedicate  my 
career  to  the  opposite.  I  wasn’t  aiming  at  anybody.  Dr. 
Dalkey’s  question  struck  me  as  being  fundamental.  I 
would  add,  however,  that  many  of  these  outstanding 
people  who  ran  into  adversity,  arose  from  a  culture 
which  encouraged  the  kind  of  activity  they  were  doing. 
In  Athens  everybody  discussed  philosophy. 

Dr.  Taylor.  We  almost  used  as  one  of  the  possible 
flues  to  creativity:  how  much  trouble  the  ide^t  cost; 
and  the  other  is  that  we  are  finding  we  do  not  have, 
at  least  on  a  broad  scale,  highly  favorable  comment  on 
creativity  in  many  of  our  organizations,  including 
schools.  However,  we  are  trying  to  do  all  we  can  in  our 
studies  and  enlighten  more  people.  I  will  be  speaking 
in  Colorado  tomorrow  and  Oregon  the  next  day  to 
illustrate  my  attempts  to  awaken  and  open  the  thing 
so  more  people  who  have  talent  can  come  from  the 
grassroots  when  they  are  ready.  The  youngsters  are 
ready  and  are  delighted  when  you  try  something  new. 

Colonel  Vaughn.  I  am  interested.  Professor  Taylor, 
in  whether  you  have  any  information  or  knowledge  of 
what  other  nations  and  cultures  are  doing  or  have  done 
rather  recently,  keeping  in  mind,  of  course,  have  the 
Russians  a  program  for  selection  of  experts  and  what 
is  it  and  how  well  is  it  working? 

Dr.  Taylor.  Let’s  go  into  the  area  of  creativity  not 
into  planning.  We  undoubtedly  have  hy  far  the  most 
research  and  most  research  knowledge  of  insight  of 
creativity  on  the  globe.  I  am  writing  a  paper  to  go  to 
Holland  next  summer  as  a  position  paper  on  creativity. 
I  was  chosen  as  the  one  in  the  world  to  write  it  and 
have  had  correspondence  with  many  of  these  places  and 
find  that  we  are  way  ahead.  That  doesn’t  mean  we  are 
way  ahead  in  practice,  the  kind  of  things  he  is  talking 
about  necessarily,  the  culture  that  is  most  favorable,  but 
at  least  ii'  the  research  area,  we  are  way  ahead.  I 
don’t  knov,  of  anyone  that  is  doing  much  in  other  high 
level  talent  areas  other  than  us.  I  think  we  are  probably 
way  ahead  there,  too.  I  have  pondered  a  little  on  cre¬ 
ativity  when  I  received  correspondence  from  the  right 
places:  Warsaw,  Moscow,  and  such,  and  way  before 
some  of  our  nations  awakened  back  in  1955,  1957,  and 
1959  whether  we  send  everything  to  them  for  fear  they 
might  implement  it  faster  than  we  did.  Our  thoughts  in 
creativity  now  is  to  shoot  the  works — send  them  any¬ 
thing  we  have.  It  will  be  one  of  the  hottest  things  they 
ever  had  in  their  hands. 

Mr.  Friedman.  If  I  can  attempt  to  translate  what  you 
were  talking  about  in  ancient  Greece  when  people  were 
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talking,  philosophizing,  from  this  some  people  emerged 
who  were  not  primarily  accepted.  Let  me  turn  that  to 
today  in  modem  times  and  point  to  the  beatniks,  the 
hippies,  who,  I  think  represent  perhaps  a  throwback  in 
modem  times  to  those  ancient  Greeks  who  sat  around 
philosophizing.  How  does  our  society  prepare  to  accept 
and  look  upon  those  hippies  today?  I  think  there  is  a 
lot  of  talent  and  creativity  in  that  group.  How  are  they 
accepted? 

Dr.  Taylor.  If  you  think  of  two  circles,  conformity 
and  creativity,  they  overlap  10  percent.  Nonconformity 
and  creativity  overlap  about  10  percent  and  among  the 
nonconformists  you  might  find  some  creatives  but  you 
might  find  many  others  who  are  different.  We  spent 
some  time  this  summer  with  Toynbee —  he  writer  his¬ 
tory  in  terms  of  creating  and  emerging  and  talks  about 
the  two  stages  of  creativity.  Maybe  this  pattern  isn’t 
always  followed  but  one  is  called  withdrawal  and 
the  other  return.  He  talks  about  this  establishment  as 
tending  to  be  conserving,  conserving  what  we  have. 
John  Gardner  has  said:”  Too  many  people  in  leader¬ 
ship  positions  spend  all  their  energies  attending  the  old 
operations  and  doing  nothing  to  improve  operations.” 
Then  people  who  are  disenchanted  or  can’t  quite  take 
what  we  have  in  the  establishment  withdraw  sensing 
something  is  not  perfect,  but  can  be  improved,  and 
they  react  sufficiently  in  disenchantment  that  they  break 
away  and  withdraw,  ifou  raised  the  question  about  the 
hippies.  If  the  second  stage  is  going  to  return  or  create 
a  whole  new  society — he  usually  thinks  of  history  as 
their  returning  and  modifying  the  establishment  suf¬ 
ficiently  to  make  history — he  is  raising  the  question; 
Will  full  scale  creativity  come  out  of  this?  TTiey  are 
showing  the  first  sign — they  are  a  product  of  this,  a 
by-product,  and  are  not  satisfied  with  the  rat  race. 
What  is  it  they  say?  Get  out  of  your  mind  and  come  to 
your  senses.  Whether  they  will  be  able  to  come  back 
and  tackle  the  establishment  successfully  to  make  some 
change  in  the  establishment  is  the  question.  The  point 
I  was  trying  to  make,  in  response  to  Dr.  Manz’  com¬ 
ment,  is  not  necessarily  the  university,  faculty  or  the 
student;  but  perhaps  the  outside,  the  fringe  society 
around  the  university,  political,  social,  whatever  it  may 
be,  that  is  resisting  change.  Perhaps  of  teaching 
methods  or  what  have  you,  more  than  the  students,  or 
the  faculty.  The  constraint  here  is  not  with  the  uni¬ 
versity  and  teaching  methods  necessarily  but  that  soci¬ 
ety  which  surrounds  the  university  and  resists  change 
within  the  university. 

Mr.  Irwin.  I  have  a  feeling  from  listening  to  ’he 
responses  that  the  solution  is  not  in  changing  the  cur¬ 
riculum  but  the  other  factors  surrounding  it.  If  we  go 


ahead  and  devise  a  better  curriculum  we  are  not 
necessarily  solving  the  problem. 

Dr.  Manz.  I  agree  as  far  as  curriculum  goes.  A  few 
days  ago  my  son  came  home  and  had  to  memorize 
symbols  of  the  periodic  system  of  the  elements.  1  re¬ 
member  when  I  was  in  school,  I  learned  it  because  I 
loved  it.  I  learned  it  without  knowing  it  but  I  never  had 
to  sit  down  and  memorize  it.  When  a  teacher  doesn’t 
know  what  he  is  doing — he  is  doing  severe  damage  to 
talented  people  having  them  memorize  completely  dead 
information. 

Mr.  Cetron.  You  mentioned  the  nonconformists  are 
only  partly  overlapping.  How  many  individuals  are 
way  ahead? 

Dr.  Taylor.  You  want  creativity  to  emerge.  Evidence 
from  the  studies  of  highly  creative  people  is  that  we 
almost  have  to  go  back  to  energy  use.  If  we  want  to  do 
something  with  minimum  expenditure  of  energy,  we  get 
a  habit  working  for  us  so  we  can  tie  our  shoes  without 
thinking — do  you  understand  what  I  am  saying? — or 
we  don’t  pay  attention  to  certain  things.  We  don’t 
expend  any  energy.  The  highly  creative  people,  accord¬ 
ing  to  studies  today,  are  rather  typical  people  in  many 
areas  using  habitual  responses  and  so  on  and  con¬ 
serving  their  energies,  a  high  percentage  are  available 
for  areas  of  battle,  so  to  speak.  Does  that  make  sense? 
They  would  appear  to  be  conforming  according  to 
many  things  and  quite  a  few  are  trying  to  do  all  they 
can  to  be  nonconforming  or  conforming  to  a  noncon¬ 
formist  pattern. 

I  said  there  are  killers  of  creativity.  I’m  not  sure  I 
agreed  with  that  last  comment.  I  think  lower  down  in 
the  school  system,  the  greater  hope — the  university  can 
be  our  hardest  nut  to  crack.  1  also  think  the  excuse  of 
the  faculty  and  principals  and  so  on  is  the  public 
because  of  their  lack  of  leadership.  The  public  wants 
the  best  it  can  get  for  their  kids.  Take  the  public — ^they 
are  with  you  if  you  can  give  them  the  best — ^this  is 
from  the  research  we  have  on  the  youngsters. 

I  made  a  speech  at  the  PTA  that  almost  ail  children 
are  above  average,  and  almost  a  third  of  them  are 
highly  gifted  in  at  least  one  area  of  talent.  The  public 
and  the  children  will  be  with  us  when  the  system  is 
better  adjusted.  It  is  the  system  that  had  adjustments 
to  make.  Let  me  close  with  this  story:  It  is  an  anti¬ 
killer,  a  killer  like  an  antimissile  missile.  There  are 
many  killers  of  creativity  around.  It  happens  so  natur¬ 
ally  and  humanly  that  it  just  kills  you  to  see  it  happen 
especially  to  you.  Someone  said :  what  if  we  decide  this 
is  a  highly  important  talent  in  creativity  and  we  should 
do  all  we  can  to  identify  and  develop  it  in  our  children 
for  the  next  15  years,  and  after  we  did,  would  there  be 
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too  many  creative  people?  The  reply  was  made;  If,  at  problem,  there  will  be  enough  creative  people  around 

that  time,  there  are  too  many  creative  people  and  it  is  a  to  solve  it. 


In  principle,  the  demographer  deals  with  a  simple  item,  the  individual  human  being, 
who  in  principle  should  be  easily  countable.  However,  Mr.  Irwin’s  paper  shows  that 
this  simplicity  is  misleading.  The  actual  counting  of  those  humans  who  are  now  alive 
is  a  serious  enough  problem,  let  alone  predicting  where  they  will  move,  when  they  will 
have  children,  and  when  they  will  die.  This  discussion  of  the  problems  of  demographers 
should  be  comforting  to  those  forecasters  who  have  to  deal  with  even  more  intractable 
data. 

— ^Editor 
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DEMOGRAPHIC  PROJECTION  TECHNIQUES 


Donald  5.  Akers  and  Richard  Irwin,  Bureau  of  the  Census,  Department  of  Commerce 


INTRODUCTION 


Those  who  make  projections  of  population  and  of  the 
demographic  characteristics  of  population  have  made 
little  use  of  the  methods  developed  by  economists  and 
by  others  engaged  in  forecasting.  Rather,  they  have 
developed  their  own  techniques,  in  response  to  the 
character  of  vital  statistics  and  population  census  data. 
This  paper  discusses  some  of  those  techniques. 

The  study  of  population  dynamics  has  a  long  history 
of  which  the  development  of  the  life  table  in  the  19th 
century  and  of  stable  population  theory  later,  in  the 
1920’s,  are  notable  examples.  It  has  some  similarities 
to  the  study  of  depreciation  of  physical  assets,  but  in 
general  it  has  unique  features  not  common  to  other 
disciplines.  A  population  is  composed  of  a  finite  num¬ 
ber  of  units  (human  beings)  each  progressing  from 
age  to  age  through  time,  beginning  with  birth  and 
ending  with  death.  Thus  age  is  central  to  most 
demographic  projections. 

Although  once  it  was  common  to  develop  population 
projections  by  fitting  curves  to  the  total  population, 
now  it  is  usual  to  analyze  growth  by  components  of 
change,  by  age,  or  by  age  and  components  of  change 
combined.  Population  change  is  the  net  sum  of  births, 
deaths,  and  in-  and  out-migration.  Projections  may  be 
made  of  each  component  separately  and  total  change 
then  obtained  by  summation.  The  presumption  is  that 
the  determinants  of  the  number  of  births,  deaths,  and 
migrants  are  independent  of  each  other.  This  is  not 
entirely  true,  but  true  enough  for  statistical  purposes. 

The  projections  are  further  defined  by  making  pro¬ 
jections  for  each  age  group  in  the  population  separately 
and  obtaining  total  population  by  summing  all  ages. 
Each  age  group  is  at  a  different  point  in  the  life  cycle. 
To  keep  this  concept  of  a  life  cycle  straight,  we  define 
an  age  cohort.  An  age  cohort  is  all  the  population  born 
in  the  same  year  or  group  of  years.  The  cohort  assumes 
different  ages  in  different  years  as  it  passes  through 
life.  For- example,  the  cohort  born  in  the  years  1895 
through  1899  was  age  60-64  in  1960  and  will  be  70-74 
in  1970. 


In  order  to  provide  for  an  orderly  progression  of 
cohorts  from  age  to  age,  the  age  detail  and  the  time 
detail  must  have  the  same  interval.  Thus,  we  must  make 
projections  of  5-year-age  groups  by  5-year  intervals  of 
time  and  single  years  of  age  by  single  year  intervals  of 
time. 

Let  us  review,  at  this  point,  how  population  projec¬ 
tions  are  made  by  what  may  be  called  the  cohort -com¬ 
ponent  method.  One  starts  with  a  population  at  some 
base  date  distributed  by  age  and  sex.  Then  one  multi¬ 
plies  the  number  of  women  in  the  childbearing  ages 
by  age-specific  birth  rates  to  determine  the  number  of 
births  in  the  subsequent  period.  One  multiplies  the 
population  and  births  by  survival  factors  to  determine 
the  number  of  survivors  at  the  end  of  the  period. 
Finally  one  adds  in  the  number  of  net  migrants  td.  the 
survivors.  The  result  is  an  estimate  of  population  by 
age  and  sex  advanced  one  interval  of  time  and  one 
interval  of  age. 

In  organizing  a  project  calling  for  population  pro¬ 
jections  of  one  or  more  subgroups  of  a  given  popula¬ 
tion,  a  common  procedure  is  first  to  develop  a 
population  projection  for  the  largest  unit  under  con¬ 
sideration,  called  here  the  parent  population.  This 
parent  population  is  commonly  the  population  of  the 
Nation  as  a  whole  but  may  be  the  population  of  a 
Slate  or  local  area  or  boit'^  subgroup  such  as  American 
Indians  or  the  population  living  in  households.  The 
population  may  then  be  distributed  by  area,  such  as  by 
State,  or  by  characteristics  such  as  school  attendance, 
marital  status,  or  labor  force  participation.  The  pro¬ 
jections  for  the  parent  population  should  be  more 
accurate  than  for  the  subgroup,  for  the  smaller  unit  is 
subject  to  all  the  factors  of  change  that  the  parent 
population  is  plus  the  factors  that  lead  to  redistribution 
of  population  among  the  subgroups.  The  procedure  of 
estimating  the  parent  population  first  permits  the 
separation  of  the  analysis  of  population  change  into 
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lli08e  factors  coounon  to  all  subgroups  *nd  those 
faclon  canrag  redistribution  between  them. 

This  paper  will  first  discuss  the  problem  of  making 


national  population  projections,  next,  of  making  pro¬ 
jections  of  demographic  characteristics,  and  then  of 
making  State  and  local  area  projections. 


NATIONAL  PROJECTIONS 


In  making  projections  of  the  national  population, 
projections  are  first  made  of  birth  rates,  death  rates, 
and  net  immigration  distributed  by  age  and  sex.  It  is 
believed  that  mortality  and  immigration  may  be 
estimated  with  fair  precision. 

Internal  migration  may  be  the  most  important  com¬ 
ponent  for  meny  subareas  of  the  population,  but  immi¬ 
gration  is  of  only  secondary  importance  to  the  growth 
of  the  United  States  population.  Over  the  past  10  years 
it  has  amounted  to  about  13  percent  of  total  growth. 
Furthermore,  it  is  a  fairly  stable  component.  Tl»e  quota 
is  the  most  important  determinant  of  annual  immigra¬ 
tion.  There  are  many  other  categories  of  migrants  to  be 
considered  such  as  arrivals  from  Puerto  Rico,  illegal 
immigration  from  Mexico,  refugees  from  Cuba  and 
elsewhere,  citizens  who  choose  expatriation,  and  the 
arrival  of  children  bom  abroad  of  American  parents. 
None  of  these  are  important  enough  to  change  the 
general  picture  of  limited  immigration.  The  most  im¬ 
portant  source  of  error  for  the  projections  is  the  age 
and  sex  distribution  assigned  to  the  immigrants. 

Mortality  is  now  low,  and  except  for  the  possibility 
of  war  or  national  disaster  should  not  vary  much.  It 
cannot  be  expected  to  drop  much  more,  for  it  is  now 
close  to  some  kind  of  biological  limit  dictated  by 
human  longevity.  Nor  is  there  any  reason  to  believe 
that  mortality  will  increase.  There  are  two  schemes  for 
projecting  mortality  rales.  One  is  to  extrapolate  past 
trends  and  the  other  is  to  set  some  kind  of  ultimate 
level  for  mortality  based  on  an  analysis  of  specific 
cau.se  of  death.  In  the  second  scheme,  period  rates  are 
obtained  by  interpolation  between  the  present  and 
ultimate  levels. 

It  is  the  births  that  make  population  projections  for 
the  United  States  so  '.mcertain.  fn  the  last  50  years, 
births  have  fluctuated  widely  and  proven  quite  un¬ 
predictable.  This  may  be  seen  in  figures  lll-l  and 
II1-2  which  show  the  total  nunflier  of  births  and  the 
total  fertility  rate  (a  measure  of  births  per  woman) 
from  1925  to  1990.  TTfie  charts  show  wide  fluct  nation  in 
paat  births  and  a  wide  band  in  expected  births.  They 
suggest  s  definite  cyclic  pattern  of  about  .3.5  years 
duration  with  peaks  in  l‘;22,  1957,  and  1970-aj  and 
Irougba  bi  1932  and  1067-70  These  fluctuati  ons  have 
been  analyzed  by  Whelpt  on,  Ryder,  and  others,  but  so 


far,  their  analysis  has  not  added  greatly  to  the  precision 
of  the  projections. 

It  should  be  noted  that  the  swings  in  births  do  not 
follow  the  swings  in  economic  activity.  There  have 
been  marked  declines  in  fertility  in  the  1920’s  and  the 
1%0’s  despite  economic  prosperity  in  those  years. 
Births  do  follow  the  4-year  business  cycle,  but  these 
short  term  swings  add  little  to  the  total  variance  in 
fertility.  There  is  some  evidence  that  young  adults  have 
not  fully  shared  the  prosperity  of  the  lOoO’a.  However, 
it  is  diflicult  to  convert  this  r!,servation  into  a  forecast 
of  future  fertility.  It  would  not  seem  that  economic 
forecasts  are  useful  for  making  national  population 
projections. 

Fertility  may  be  disaggregated  analytically  into 
average  completed  size  of  family  and  the  age  at  which 
women  bear  their  children.  (The  age  at  which  women 
bear  children  is  referred  to  as  the  timing  of  births.) 
Fluctuations  in  completed  size  of  family  are  measured 
by  the  completed  fertility  rate  shown  in  figure  III-3. 
The  chart,  shows  that  size  of  f.  ,ily,  shown  by  the  com¬ 
pleted  fertility  rate,  has  flucti.ated  less  than  annual 
fertility  shown  by  the  total  fertil'ty  rate.  The  rest  of 
the  change  must  be  explained  by  changes  in  age  of 
mother  at  childbearing.  Four  distributions  of  age  of 
mother  are  shown  in  figure  III-^.  Series  A  is  an  ap¬ 
proximation  of  the  experience  of  the  cohort  of  women 
born  in  1930-40,  who  reached  the  peak  of  childbearing 
in  I960;  Series  D  is  an  approximation  of  the  experi¬ 
ence  of  the  cohort  of  women  born  in  1910-29  who 
reached  the  peak  of  childbearing  in  H>40,  It  is  evident 
that  historically  high  fertility  has  been  associated  with 
young  average  age  of  mother  and  low  fertility  with  an 
older  average  age  of  mother.  'Fhesc  changes  in  average 
age  of  childbearing  may  be  measured  by  the  median 
age  of  chillibearing  shown  in  figure  III  -5.  It  .shows  that 
the  median  for  the  1*)10  20  cohort  (used  for  Series  D) 
was  about  27.2  years  and  the  meslian  for  the  1930-^10 
cohort  I  used  for  Series  A )  was  alwiut  25. .3  years. 

■Age  of  mother  at  childJiearing  affects  fertility  in  two 
wavs.  In  the  long  run,  it  determines  the  length  of  gen¬ 
eration.  The  lower  the  meilian  age  of  mother,  the 
shorter  is  the  length  of  generation,  and  eonsequentlv. 
the  higher  is  the  projeited  nuini>er  of  births.  In  the 
shortrun.  charigr->  in  age  of  mother  cause  birtlis  to 
bunch  up  or  fan  out. 
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NOTE:  DISTRIBUTIONS  RELATE  TO  COHORTS  BORN  AFTER  JULY  1951. 


Fii.i/bt  LI-4.  Distribution  of  Age-s’'ecific  Birth  Rates  Associated  With  Termiriol  Completed  Fertility  Rates. 


In  the  1950’s  women  w.io  had  delayed  having 
children  during  tlis  depressioi:  and  war  years  were 
having  children  at  a  relatively  high  rate  in  their  later 
childbearing  years.  Thus,  the  1920  cohort  had  a  high 
median  age  i'  ’ildbearing.  At  the  same  time  the 
younger  womt  .e  having  their  children  at  a  young 
age.  Thus,  the  1930-35  cohoitr  had  a  lower  median 
age.  The  high  fertility  of  the  1950’  is  to  be  explained 
in  part  by  this  overlapping  of  cohort  fertility,  a  situa¬ 
tion  that  could  not  persist.  The  downward  slooe  from 
die  1920  to  the  1940  cohort  in  figure  III-5  is  sympto¬ 
matic  of  this  overlap.  Today,  the  situation  is  reversed. 
Older  women  are  having  fewer  children  and  younger 
women  are  delaying  having  their  fatnilies.  This  is 
illustrated  once  more  by  the  upward  slope  after  the 
19  ■'0  cohort.  ■  ? 

It  may  seem  from  this  analysis  that  changes  in  age 
of  mother  causing  peaks  and  trtmghs  is  fertility  exceed 
the  range  of  variability  expected  for  completed  size  of 
familv.  Disentangling  these  fluctuations  from  the  his¬ 
torical  data  an^  deciding  whether  to  build  the  cycles 
back  into  the  projected  series  are  two  problems  to  be 
resolved  in  projecting  births. 


Two  methods  of  making  fertility  projections  that  take 
account  of  this  complex  picture  are  being  tested — the 
cohort-fertility  method  and  the  parity-progression 
method.  The  cohort-fertility  method,  as  the  name  im¬ 
plies,  analyzes  fertility  in  terras  of  cohort.  It  assigns  an 
average  size  of  completed  family  to  each  cohort.  It  then 
distributes  by  age  mother  the  differences  between 
the  completed  size  and  the  cumulative  fertility  already 
experienced  by  the  cohort.  The  age-specific  fertility 
rates  thus  generated  are  regrouped  into  an  annual 
series  and  then  fed  into  population  projections  of  the 
cohort-component  variety  described  above.  The  cohort- 
fertility  method  is  the  one  that  was  used  for  preparing 
the  latest  set  of  population  projections  by  ihe  Bureau 
of  the  Census. 

The  parity-progression  techni'rue  starts  with  v.’omen 
of  childbearing  age  at  some  base  date  distributed  by 
age,  marital  status,  parity  (that  is,  the  number  of 
children  ever  born  to  the  woinan),  and  interval  since 
last  child  (or  since  marriage  for  childless  women).  The 
women  are  multiplied  *  y  a  corresponding  set  oi  birth 
probabilities.  This  operation  determines  both  the  num¬ 
ber  of  births  and  the  progress  of  women  from  one 
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■Median  Age  of  Mother,  by  Birth  Cohort  of  Woman:  Birth  years,  1900-01 — 195d-S5. 


parity  to  the  next.  If  mean  parity  is  high  at  the  base 
^  period,  a  peak  has  been  passed  and  fertility  should 
drop;  if  mean  parity  is  low  a  trough  has  been  passed 
and  fertility  should  rise.  Interval  between  births  is  a 
measure  of  timing  of  births  and  determines  age  of 
mother.  There  has  been  less  testing  of  the  parity- 
progression  method  than  of  the  cohort-fertility  method. 

One  distinction  between  these  two  methods  is  similar 
to  one  already  made  in  discussion  mortality  projec¬ 
tions.  The  parity-progression  method  is  an  extrapola¬ 
tion  of  rates  of  the  recent  past  whereas  the  cohort 
fertility  method  starts  with  an  assumption  about  the 
ultimate  level  of  fertility  and  works  backward. 

The  success  of  the  cohort-fertility  method  depends  in 
part  on  the  successful  pred.’clion  of  completed  size  of 
family.  Ryder  and  Lawrence  have  suggested  methods 
of  doing  this  by  mathematical  extrapolation.  However, 
more  reliance  has  been  placed  on  the  results  of  national 
surveys  designed  to  measure  expected  size  of  family 
by  interviewing  women  in  the  childbearing  ages. 
Whelpton  and  Campbell  believe  that  the  expectation 
data  are  highly  predictive.  Critics  say  that  women  can¬ 
not  really  assess  all  the  future  developments  that  may 
affect  fertility.  They  point  out  that  within  5  years  half 
the  births  will  occur  to  women  who  were  young  and 
unmarried  at  the  time  of  the  survey  and  thus  fall  out¬ 
side  its  scope.  Finally,  they  object  to  a  philosophy  that 
says  average  expectations  are  predictive  even  when  :t 
appears  that  expectations  for  individual  women  are  not. 

The  average  size  of  family  expected  by  the  women 
surveyed  has  remained  very  stable  in  the  last  few  years 


despite  the  sharp  drop  in  fertility.  It  may  be  argued 
that  the  drop  is  due  entirely  to  a  change  in  timing,  and 
not  to  a  tendency  for  women  to  have  smaller  families. 
Experimental  calculations  show  that  this  is  possible. 
If  this  is  true,  a  rebound  of  fertility  is  likely.  However, 
it  is  hard  at  this  point  to  give  full  credence  to  the 
surveys. 

The  latest  projections  made  by  the  Bureau  of  the 
Census  use  the  results  of  the  surveys  only  in  a  general 
way.  Three  different  surveys  all  place  average  com¬ 
pleted  fertility  at  about  3  children  per  woman.  The 
projections  assume  a  range  of  2.45  to  3.35  children 
per  woman.  They  also  vary  the  tuning  of  births.  As  a 
result  the  projections  have  a  greeter  range  in  the  short 
than  in  the  longnin.  We  are  hopeful  that  the  parity- 
progression  method  will  give  more  reliable  shortrun 
projections,  although  probably  less  reliable  longrun 
projections,  than  does  the  cohort  fertility  method. 
Perhaps  eventually  we  will  find  some  way  to  splice  the 
two  methods  so  as  to  obtain  the  advantages  of  both. 

There  is  great  uncertainty  as  to  level  of  future  fer¬ 
tility.  Despite  the  considerable  current  research  that 
has  gone  into  the  subject,  much  needs  to  be  done.  There 
is  a  need  for  a  great  deal  of  mechanical  experimenta¬ 
tion  with  the  proposed  methods  for  making  the  pro¬ 
jections.  There  is  a  need  for  more  detailed  data  on 
birth  interval  for  tlie  parity-progression  method.  There 
is  a  need  to  enlarge  and  refine  the  surveys  on  birth  ex¬ 
pectations,  and  there  is  a  special  need  to  consider  the 
philosophical  underpinnings  for  using  expectation  data 
for  predicting  the  future. 


DEMOGRAPHIC  CHARACTERISTICS 


So  far,  we  have  been  discussing  projections  of  popu¬ 
lation  only.  We  may  now  consider  methods  of  project¬ 
ing  demographic  characteristics.  These  include  school 
enrollment,  educational  attainment,  labor  force  par¬ 
ticipation,  marital  status,  and  the  number  of  households 
and  families. 

There  are  two  general  methods  of  projecting 
demographic  characteristics.  One  is  the  cohort-com¬ 
ponent  method,  already  discussed  in  explaining  the 
manner  of  projecting  the  total  population.  The  other  is 
the  participation  rate  method.  In  this  method,  projec¬ 
tions  are  made  of  the  proportion  of  the  population  at 
each  age  in  some  subgroups  of  the  population.  These 
proportions  are  then  applied  to  the  total  population  by 
age  to  obtain  the  number  in  that  subgroup.  For 
example,  projections  may  be  made  of  labor  force  par¬ 
ticipation  rates  by  age,  and  these  rates  are  then  applied 


to  the  total  population  to  obtain  the  number  in  the 
labor  force. 

Let  us  give  two  examples;  projections  may  be  made 
of  school  enrollment  rates  by  age,  and  these  rates 
applied  to  projections  of  the  total  population  by  age 
to  obtain  the  number  enrolled.  This  is  the  participation- 
rate  method.  Or,  grade  progression  rates  may  be 
applied  to  the  school  population  distributed  by  grade 
at  some  base  date  to  project  the  number  left  in  school. 
This  is  the  cohoi  (-component  method. 

The  second  example  is  of  marital  status.  Projections 
may  be  made  of  the  jrercent  single  by  age,  and  these 
percentages  are  then  applied  to  projections  of  the  total 
population  to  determine  the  distribution  between  single 
and  ever-married.  This  is  the  participation  rate  method. 
Or;  first-marriage  rate"  may  be  applied  to  the  single 
population  at  some  base  date  to  determine  the  number 
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of  marriages.  Then  first  marriages  are  subtracted  from 
the  single  population  and  added  to  the  ever-married 
population  to  determine  the  marital  status  of  the 
population  at  the  end  ot  ihe  year. 

Each  method  has  advantages  and  drawbacks.  One 
consideration  is  whether  there  is  likely  to  be  a  cohort 
effect,  that  is  whether  the  previous  experience  of  a 
cohort  exerts  a  significant  influence  on  its  future  de¬ 
velopment.  For  example,  a  cohort  that  has  unusually 
high  enrollment  part  way  through  the  education  cycle 
is  likely  to  continue  having  high  enrollment  thereafter 
and  to  achieve  high  attainment  in  the  future.  This 
information  will  be  lest  if  the  projec'^ions  are  made  by 
the  participation-rate  method.  On  the  other  hand,  the 
cohort-component  method  is  affected  unduly  by  errors 
in  the  data  for  the  population  at  the  base  date  and  will 
accumulate  errors  through  the  repeated  application  of 
the  transition  rates  to  the  population.  Where  pro¬ 
jections  are  made  by  a  linked  process,  as  in  the  cohort- 
component  method,  errors  in  the  data  tend  to  cumulate 
until  unreasonable  results  may  be  obtained.  Cumulative 


errors  may  be  reduced  if  a  target  can  be  imposed  on 
each  cohort.  For  example,  the  proportion  ultimately 
marrying  might  be  assigned  to  each  cohort  in  the 
marital  projections. 

A  suitable  method  for  projecting  households  depends 
on  the  state  of  the  housing  market.  If  housing  is  in 
short  supply  as  is  still  true  in  many  European  countries, 
the  number  of  households  will  depend  on  the  enterprise 
of  the  construction  industry.  If  they  are  in  plentiful 
supply,  then  the  number  will  depend  on  the  numbe  of 
adults  who  decide  to  establish  their  own  households. 
Since  the  number  of  households  equals  the  number  of 
household  heads,  the  latter  may  be  projected  like  any 
other  demographic  characteristic.  Theoretically,  either 
the  cohort-component  or  the  participation-rate  tech¬ 
nique  n’By  be  employed,  but  the  data  needed  for  input 
to  the  cohort  component  technique  are  not  readily 
available.  Hence  in  practice,  as  with  most  other  demo¬ 
graphic  characteristics,  the  participation-rate  method  is 
more  frequently  used. 


LOCAL  PROJECTEONS 


Projections  for  States  and  local  areas  may  be  made 
by  extrapolating  past  trends  in  population  growth,  by 
estimating  the  response  to  some  future  development, 
such  as  the  location  of  new  industries,  or  a  , change  in 
the  relative  attractiveness  of  different  geographic  areas 
as  places  of  residence.  Often  there  is  little  really  known 
about  the  determinants  of  future  growth,  so  that  some 
kind  of  extrapolation  of  past  trends  by  component  is 
the  most  commonly  used  method.  The  procedure  is 
strengthened  by  reference  to  projections  for  a  parent 
population  for  the  birth  and  death  components. 

In  developing  projections  of  population  or  of  popu¬ 
lation  characteristics  for  a  single  area  such  as  a  State 
or  county,  a  choice  can  be  made  between  developing 
a  projection  independent  of  outside  factors,  i.e.,  de¬ 
veloping  a  parent  population,  or  treating  the  area  as  a 
subgroup  of  a  larger  unit  for  which  projections  already 
exist. 

If  the  State  or  local  area  is  treated  as  a  parent 
population  the  problems  faced  are  similar  to  those  for 
national  projections,  except  that  the  component  of 
migration  is  relatively  much  more  important,  and  often 
insufficient  data  are  available  to  determine  historical 
trends  in  migration  accurately.  In  projecting  fertility, 
only  period  fertility  analysis  can  be  used,  because 
migration  obliterates  cohort  fertility  analysis,  and  the 
data  needed  for  parity  progression  techniques  are  not 
available. 


If  the  area  is  treated  as  a  subgroup  of  a  parent 
population,  a  common  procedure  is  to  restate  basic 
data  in  terms  of  their  proportion  to  the  parent  popula¬ 
tion.  A  historical  series  of  these  proportions  may  be 
extrapolated  into  the  future.  This  proportion  is  then 
multiplied  by  the  projected  figure  for  the  parent  popu¬ 
lation  to  determine  the  desired  figure  for  the  area. 
This  technique,  usually  referred  to  as  the  ratio  method, 
is,  of  course,  related  to  the  participation-rate  method 
described  above.  The  ratio  method  can  be  developed  in 
widely  varying  degrees  of  detail.  A  single  projected 
total  population  for  a  single  area  can  be  obtained  in 
this  manner,  or  a  more  detailed  projection  for  all  areas 
can  be  obtained.  It  may  also  be  applied  to  each  age 
group,  with  the  total  population  obtained  by  sum¬ 
mation.  The  procedure  is  simple  and  assures  a  d^ree 
of  consistency  with  the  parent  population. 

Even  if  the  cohort-component  m^hod  is  used,  effec¬ 
tive  reference  to  a  parent  population  can  be  retained, 
especially  if  projections  for  all  subgroups  of  the  parent 
population  can  be  developed  simultaneously.  This  is  the 
procedure  used  by  the  Bureau  of  the  Census  in  develop¬ 
ing  population  projections  for  States.  To  project 
deaths,,  one  national  set  of  age-sex-color  specific  sur¬ 
vival  rates  has  been  used,  and  the  sum  of  deaths  for 
all  States  derived  in  this  manner  is  forced  into  agree¬ 
ment  with  the  number  of  deaths  developed  by  the 
national  population  projections. 
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Fertility  rates  are  also  linked  to  national  values, 
although  rates  specific  to  each  State  are  employed.  The 
ratio  of  each  State’s  general  fertility  rate  to  the  national 
ratw  at  the  base  date  is  computed,  and  projected  values 
are  obtained  by  assuming  convergence  to  national  rates 
at  aotne  future  date.  Values  for  intermediate  projection 
dates  are  obtained  by  interpolation.  The  births  are 
generated  by  multiplying  the  projected  fertility  rates  by 
the  appropriate  population  at  any  future  date.  Births 
are  then  summed  for  all  States  and  forced  prorata  into 
agreement  with  births  projected  for  the  Nation. 

The  component  offering  the  highest  degree  of  un¬ 
certainty  in  State  and  local  projections  is  migration. 
Although  an  error  in  the  projection  of  the  birth  com¬ 
ponent  over  a  10-year  period  could  cause  a  deviation 
of  up  to  10  percent  from  the  total  population  that 
actually  develops,  the  error  would  tend  to  be  of  the 
same  sign  for  all  areas,  thus  not  changing  their  relative 
position  a  great  deal.  The  migration  component  could 
cause  a  deviation  of  from  10  to  20  percent  of  the  total 
population  or  even  more,  and  the  differences  in  the 
actual  growth  rates  from  those  projected  would  be 
accentuated  from  area  to  area,  since  the  gain  of  one 
State  through  migration  is  a  loss  to  the  others.  Further¬ 
more,  net  migration  rates  are  subject  to  much  greater 
fluctuation  in  the  shortrun  than  is  the  case  with  birth 
rates.  For  diese  reasons,  considerable  attention  has 
been  given  to  the  analysis  of  internal  migration  by 
demographers,  and  by  economists  and  members  of  the 
business  community.  The  economists  and  businessmen 
are  drawn  into  this  field  by  the  desire  to  forecast 
economic  variables,  many  of  which  are  heavily 
dependent  on  population  size  and  composition. 

In  making  population  projections  migration  between 
arsas  within  a  parent  population  can  be  calculated 
either  as  n^  migration,  that  is  the  excess  or  deficiency 
of  inmigration  as  oppos-^l  to  outmigration,  or  as  gross 
migration,  where  separate  computations  are  made  for 
out-  and  inmigration. 

Under  the  gross  migration  procedure  the  sum  of  the 
projected  outmigrants  must  equal  the  sum  of  all  in- 
migrants,  and  when  net  migration  is  used  the  sum  of 
the  net  values  for  all  areas  must  be  zero.  Ihus  migra¬ 
tion  for  one  area  cannot  logically  be  calculated  in 
isolation,  for  it  must  affect  the  migration  for  some  other 
area. 

Projections  of  net  migration  have  been  more  com¬ 
mon  than  projections  of  gross  migration,  because  the 
data  for  estimating  net  migration  have  been  more 
readily  available.  Since  the  mobility  of  persons  in  the 
United  States  is  free  of  political  restrictions,  event 
rqwrting  of  migration  has  been  meager.  Net  migration. 


however,  can  be  obtained  by  noting  population  change 
from  one  census  to  another,  explaining  some  of  the 
change  by  reference  to  registered  births  and  deaths, 
and  inferring  that  the  remaining  change  is  due  to  net 
migration.  Net  migration  determined  in  this  fashion  is 
sometimes  called  residual  net  migration.  Unfortunately 
migration  values  obtained  in  this  manner  include  errors 
of  census  underenumeration,  age  misreporting,  or 
errors  in  the  registration  of  vital  statistics.  None  the 
less,  net  interstate  migration  is  often  so  heavy  as  to 
dominate  the  picture. 

Although  net  migration  statistics  are  readily  avail¬ 
able,  their  use  in  population  projections  is  not  entirely 
satisfactory.  There  are  certain  technical  difficulties 
(discussed  below)  in  balancing  the  sum  of  net  migra¬ 
tion  for  all  areas  to  zero.  More  fundamentally,  net- 
migration  represents  only  the  surface  effect  of  two 
powerful  streams  of  out-  and  inmigration.  When  one 
probes  for  causes  of  migration,  the  search  leads  not  to 
one  question,  “Why  is  the  net  effect  of  migration  thus 
or  thus?”  but  to  two  questions,  “Why  do  certain 
people  move  out,  and  others  move  in?”  For  example, 
even  in  areas  of  heavy  net  outmigration,  there  is  con¬ 
siderable  gross  inmigration,  and  in  areas  of  heavy  net 
inmigration  the  rate  of  gross  outmigration  may  be  as 
high  as  for  areas  of  net  outmigration.  Thus  gross 
migration  has  recently  come  under  increasing  scrutiny. 
If  the  necessary  basic  statistics  are  available,  gross 
migration  as  greatly  to  be  preferred  over  net  migration 
as  a  basic  projection  technique. 

Projections  of  gross  migration  can  be  handled  either 
by  calculating  the  migration  streams  between  each 
pair  of  areas,  or  by  calculating  outmigration  regardless 
of  destination,  and  inmigration  regardless  of  source. 
By  this  latter  technique  a  “migration  pool”  is  developed 
by  first  computing  outmigration  for  all  areas  and  then 
distributing  this  pool  back  to  the  areas  as  inmigrants. 

To  our  mind,  the  pool  method  of  projecting  migra¬ 
tion  is  to  be  preferred  at  this  time  over  the  place  to 
place  stream  method,  taking  into  consideration  the 
character  of  the  basic  statistics  and  the  imprecise 
nature  of  present  knowledge  about  the  determinants  of 
migration.  It  is  believed  that  outmigration  is  a  function 
of  specific  population  characteristics,  especially  age, 
and  is  relativeiv  constant  from  area  to  area,  while  in- 
migration  is  mote  sensitive  to  the  different  levels  of 
attractiveness  of  alternative  destinations.  But  the 
precise  functional  relationships  involved  are  not  knovm, 
and  it  may  be  impossible  to  determine  them  without 
more  precise  data.  If  place  to  place  migration  streams 
by  age  are  projected  a  complex  model  with  a  very 
large  number  of  cells  is  the  inevitable  result.  On  the 
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other  hand,  the  migration  pool  method  separates 
migration  into  the  two  chief  dynamic  factors  of  out- 
and  inmigration,  yet  remains  simple  enough  to  permit 
the  analysis  of  the  relationships  involved  and  to  correct 
for  shortcomings  of  the  data  with  relative  ease. 

A  number  of  problems  exist  in  connection  with  the 
basic  gross  migration  data  obtained  in  the  1960 
Census.  Only  92  percent  of  those  persons  found  in  a 
reinterview  study  to  have  been  interstate  migrants  were 
so  indicated  by  the  regular  census  results.  This  repre¬ 
sents  misclassification  of  those  enumerated  by  the 
census.  The  overall  3  percent  net  undercount  for  the 
1960  census  may  be  presumed  to  be  considerably 
heavier  for  migrants.  Of  those  enumerated,  another  3.3 
percent  of  those  who  moved  did  not  indicate  their  place 
of  former  residence,  hence  are  not  included  in  the  out- 
migrant  statistics.  Only  partial  information  was  ob¬ 
tained  about  mobility  of  members  of  the  Armed  Force, 
college  students,  and  institutional  inmates,  yet  move¬ 
ments  of  these  special  groups  often  obsgure  migration 
more  directly  related  to  basic  economic  and  social 
factors. 

When  migration  is  projected  for  all  areas  of  a  parent 
population  by  the  application  of  rates,  the  sum  of  the 
inmigrants  projected  for  the  areas  tends  to  exceed  total 
outmigrants  and  the  values  for  the  various  areas  must 
then  be  adjusted  to  zero  (or  to  a  control  total  repre¬ 
senting  net  immigration  to  the  parent  population).  As 
the  projection  proceeds,  the  imbalance  becomes  pro¬ 
gressively  larger,  requiring  larger  and  larger  adjust¬ 
ments.  This  difficulty  is  brought  about  by  a  paradox 
implicit  in  the  use  of  migration  rates.  Specifically,  for 
areas  of  net  inmigration,  which  grow  faster  than  the 
national  average,  the  computations  produce  larger  and 
larger  numbers  of  net  inmigrants.  F  or  the  outmigrant 
States,  however,  the  computed  number  of  net  out- 
migranst  increases  less  rapidly,  since  their  population 
base  is  growing  more  slowly  than  the  national  average. 
Since  the  sum  of  net  interstate  migration  for  all  areas 
must  equal  zero,  it  is  necessary  to  adjust  the  computed 
in-  and  outmigratic  n  in  some  way. 

Various  techniques  have  been  employed  to  bring  the 
computed  number  oi  migrants  into  balance.  When 
working  with  net  migration  rates  none  of  the  techniques 
available  for  this  adjustment  is  entirely  satisfactory, 
but  we  have  found  one  that  seems  quite  satisfactory  for 
the  gloss  migration  pool  method.  In  this  method,  out¬ 
migration  is  computed  from  gross  outmigration  rates 
applied  to  the  population  of  each  area  to  form  a  pool  of 
internal  migrants.  This  pool  is  distributed  as  in- 
^igrants  among  the  areas  in  proportion  to  the  distribu¬ 
tion  of  inmigrants  during  the  base  period.  Thus  each 


outmigrant  becomes  an  inmigrant,  and  the  sum  of  n^ 
migrants  for  all  areas  is  zero.  Net  immigration  to  the 
parent  population  can  be  added  to  the  migrant  pool 
for  distribution  among  the  areas,  or  handled  separately. 
Due  to  the  distinctive  character  of  net  immigration  into 
the  United  States,  the  Bureau  of  the  Census  makes  a 
separate  allocation  of  this  component  in  its  projections 
of  State  population. 

The  same  set  of  proportions  can  be  used  without 
change  throughout  the  projection  period.  This  pro¬ 
cedure  has  the  automatic  result  that  net  migration  for 
each  of  the  areas  tends  toward  a  level  common  to  all, 
as  the  projection  proceeds  through  time.  An  area  grow¬ 
ing  more  rapidly  than  the  average  because  of  net 
inmigration  tends  to  have  its  rate  of  net  inmigration 
decreased  as  its  population  base  grows.  By  contrast  the 
slower  growing  areas  fare  better  as  time  passes  since 
they  retain  a  constant  proportion  of  the  inmigrant  pool 
while  their  relative  share  of  outmigration  is  declining. 

The  tendency  toward  equilibrium  obtained  by  treat¬ 
ing  out-  and  inmigration  in  this  manner  is  in  accord 
with  the  general  assumption  that  net  migration  streams 
will  tend  to  eliminate  differentials  existing  among  the 
areas  at  the  beginning  of  the  projection  period.  K  this 
assumption  is  not  desired,  rates  of  inmigration  can  be 
developed  for  the  base  period,  and  inmigration  com¬ 
puted  for  each  projection  period,  with  the  resulting 
values  converted  to  proportions  of  overall  inmigration 
for  the  purpose  of  distributing  the  out-migrant  pool. 
The  sum  of  net  migrants  for  all  the  areas  will  still 
balance  with  the  overall  total,  but  the  migration  as¬ 
sumption  here  would  be  of  continuation  of  net 
migration  differentials  among  the  areas. 

We  have  said  that  internal  migration  is  best  pro¬ 
jected  by  the  appli^-dtion  of  rates  to  a  |)opulation  base. 
This  is  true  of  the  majority  of  migrants  who  move  as 
individuals  or  families  in  response  to  economic  oppor¬ 
tunity  or  social  factors.  It  is  not  true  for  certain  large 
groups  of  migrants  who  move  in  response  to  admin¬ 
istrative  decision,  and  are  often  concentrated  in  large 
group  quarters  such  as  military  barracks,  college 
dormitories,  or  institutions.  Regardless  of  whether  the 
projection  is  in  terms  of  net  migration,  gross  migra¬ 
tion,  gross  migration,  or  place-to-place  streams,  account 
should  be  taken  of  movements  involving  the  Armed 
Forces,  and  if  possible,  college  students  and  inmates 
of  institutions.  It  is  particularly  important  to  remove 
these  groups  from  the  base  data  from  which  the  rates 
are  calculated.  The  need  to  take  account  of  these  special 
populations  becomes  more  acute  as  the  geographic 
detail  and  agj  detail  becomes  more  precise,  'fhus  at  the 
county  level,  the  presence  of  a  major  university  can 
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dominate  migration  atatistics  for  the  age  groups  in¬ 
volved.  To  leave  these  statistics  unadjusted,  thus 
allowing  them  to  be  converted  to  rates  of  the  population 
of  the  area,  can  seriously  distort  the  results  of  a  pO|<u- 
lation  projection.  Interstate  movement  of  college  stu¬ 
dents  is  not  such  a  serious  problem,  although  pro¬ 
jections  of  State  population  could  probably  be 
improved  by  adjusting  for  this  factor. 

In  the  case  of  the  Armed  Forpes,  however,  an 
adjustment  is  almost  mandatory  at  either  the  State  or 
county  level.  About  one  sixth  of  all  males  aged  20-24 
in  1960  were  in  the  Armed  Forces.  Since  a  standard 
tour  of  duty  is  about  3  years,  practically  every  member 
of  the  Armed  Forces  can  be  assumed  to  move  to  a 
different  State  in  5  years,  or  at  least  to  a  different 
county.  If  movements  involving  the  Ajrmed  Forces  are 
allowed  to  remain  in  the  base  period  'statistics  used  for 
computing  rates  and/or  proportions  of  migration,  the 
implicit  assumption  is  that  the  military  developments 
during  the  base  period,  both  for  the  area  and  for  the 
parent  population,  will  be  duplicated  during  each 
projection  period.  Thus,  if  a  new  military  base  is 
established  during  the  base  period,  au  unadjusted  pro¬ 
jection  will  build  it  anew  during  each  projection 
period,  like  a  sorcerer’s  apprentice. 

A  complete  adjustment  for  the  impact  of  military 
movements  ideally  should  allow  not  only  for  changes 
in  station  strength  of  military  bases,  but  should  also 
adjust  for  movement  between  the  civilian  and  military 
population.  Adjustment  consists  of  removing  these 
streams  from  the  statistics  for  the  base  period,  then 
making  some  assumption  about  future  size  of  the 
Armed  Forces  and  adding  or  subtracting  an  adjustment 
for  each  area  consistent  with  this  assumption  during 
the  projection  period. 

The  basic  data  required  to  make  these  adjustments 
are  difficult  or  impossible  to  obtain  in  full  detail,  and 
in  practice  it  is  usually  necessary  to  apply  a  partial 
correction.  It  is  feh  that  such  corrections  improve  the 
projections,  even  though  falling  short  of  a  perfect 
solution.  In  developing  an  adjustment  for  any  special 
population  it  should  be  remembered  that  census  migra¬ 
tion  atatistics  are  subject  to  errors  of  census  undercount 
and  misrepoiting.  The  1970  census  will  provide  im¬ 
portant  new  detail  as  to  the  migration  of  Armed  Forces 
personnel  and  their  dependents.  This  detail  will  contri¬ 
bute  to  the  understanding  of  census  misreporting  as 
well  as  providing  material  for  developing  a  more 
detailed  adjustment. 

To  derive  akemative  projections  of  the  population 
by  States,  the  different  fertility  assumptions  of  the 
national  projections  can  conveni:^ntly  be  used,  since 


changes  in  the  national  level  of  fertility  can  be  expected 
to  affect  all  the  States  to  some  degree.  Alternative 
migration  assumptions  have  been  developed  in  project¬ 
ing  both  gross  and  net  migration  for  States.  For  gross 
migration  it  has  been  possible  to  adjust  projected 
out-  and  inmigration  in  such  a  way  as  to  gradually 
approach  an  equilibrium  of  net  migration  among  the 
States  at  a  selected  future  point  in  lime.  This  provides 
an  alternative  value  to  that  obtained  under  the  assump¬ 
tion  of  continued  differentials  among  the  States.  For 
both  gross  and  net  migration  projections  the  length  of 
the  base  period  can  be  varied,  or  two  base  periods  can 
be  used,  averaging  the  migration  rates  for  each  period, 
and  using  different  patterns  of  weights  to  achieve  an 
alternative  migration  assumption. 

The  customary  assumption  that  migration  will  con¬ 
tinue  in  the  future  according  to  some  pattern  of  be¬ 
havior  in  the  past  has  been  felt  by  many  to  be  a  weak¬ 
ness  of  demographic  projections  for  State  and  local 
areas.  It  is  widely  believed  that  the  future  population 
development  of  a  community  is  heavily  dependent  on 
the  economic  potential  of  the  community,  and  that  once 
this  potential  is  determined,  the  approximate  popula¬ 
tion  level  which  can  be  maintained  is  determined. 
However,  projections  of  economic  activity  for  an  area 
are  often  an  extrapolation  of  past  trends.  There  is  little 
reason  to  believe  that  this  kind  of  projection  should  be 
more  reliable  than  projecting  population  from  past 
trends  in  population  growth.  It  may  well  introduce  an 
additional  source  of  error;  much  migration  is  not  in 
response  to  economic  opportunity. 

More  meaningful  projections  of  economic  activity 
may  be  made  from  a  careful  inventory  of  prospects  for 
the  growth  of  primary  industry  within  an  area,  espe¬ 
cially  of  those  that  produce  for  export  to  other  areas. 
This  information  is  used  to  estimate  prospects  for  sec¬ 
ondary  industry,  employment  in  secondary  industry 
and  construction,  employment  in  services,  excess  un¬ 
employment,  and  finally  the  population  dependent  on 
the  'abor  force.  This  is  known  as  Hoyt’s  economic  base 
method.  It  is  very  time  consuming  to  ap,  and  the 
increase  in  accuracy  may  only  be  moderate. 

A  projection  of  economic  activity  for  regions  can 
also  be  developed  by  working  from  a  parent  projection 
of  the  economy  and  allocating  industrial  activity  to  the 
various  areas  according  to  locational  factors,  princi¬ 
pally  the  location  of  resources  and  markets.  One  appli¬ 
cation  of  thia  method  allocates  secondary  industry  in 
the  same  manner  as  primary,  thus  abandoning  this 
dichotomy  as  the  explicit  basis  of  the  regional  projec¬ 
tions. 

It  has  not  yet  been  demonstrated  that  population  pro- 
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jectiong  dependent  upon  an  analysis  of  the  economic 
factors  are  more  reliable  than  those  based  on  purely 
demographic  factors,  and  for  the  present  demographic 
projections  are  coexisting  with  those  based  on  an  analy¬ 
sis  of  economic  factors.  This  coexistence  has  resulted  in 
some  cases  in  the  inclusion  of  both  techniques  in  one 
projection  model.  An  estimate  of  future  employment 
based  on  a  demographic  projection  assuming  no  mi¬ 
gration  is  compared  with  one  based  on  an  analysis  of 
economic  activity.  The  observed  difference  is  assumed 


to  result  in  a  net  migration  of  employed  persons,  from 
which  the  net  migration  of  all  persons  may  be  estimated. 

Finally,  with  respect  to  population  projeetione  for 
small  local  areas,  expected  land  use  has  provided  esti¬ 
mates  of  future  population.  Here  the  analysis  is  in  terms 
of  areas  available  for  residential  development  and  ex¬ 
pected  density  of  population.  Such  projections  are  often 
referred  to  as  saturation  populations  and  are  sometimes 
presented  without  indicating  the  date  at  which  this 
population  might  be  expected  to  materialize. 


i  RELIABILITY  OF  PROJECTIONS 


I  Statements  as  to  reliability  of  projections  are  much 

!  in  demand  by  users.  There  seems  to  be  no  way  to  assign 

•  a  probability  measure  to  a  set  of  projections,  however, 

f  It  is  common  to  avoid  the  issue  by  labeling  a  set  of 

!  projections  as  illustrative  or  as  true,  given  the  assump- 

t  tions.  This  practice  ignores  the  fact  that  some  types  of 

;  projections  would  seem  to  be  more  soundly  based  and, 

t  hence,  worthy  of  greater  confidence  than  others. 

!  One  way  of  indicating  reliability  is  to  calculate  a 

f  range.  It  is  defined  by  a  high  and  low  series  of  projec¬ 
tions.  They  are  generated  by  choosing  assumptions  that 
are  considered  extreme,  but  nevertheless,  thoroughly 
reasonable.  Thus,  the  projections  are  something  less 
,  than  confidence  limits.  Yet  the  narrowness  of  the  range 
does  give  some  indication  of  the  confidence  placed  in 
them. 

j.  The  Bureau  of  the  Census  publishes  four  sets  of 

national  population  projections.  Each  set  has  a  different 
assumption  about  fertility  but  all  sets  use  the  same 
assumptions  about  mortality  and  immigration.  Hence, 
the.  projections  differ  for  the  cohorts  yet  to  be  bom  but 
not  for  the  cohorts  already  alive  at  the  base  date.  There 
is  an  implication  here  that  the  projections  for  these 
cohorts  are  so  reliable  that  there  is  no  need  to  indicate 
the  degree  of  reliability  by  a  range.  Certainly,  they  are 
more  reliable  than  the  projections  for  the  cohorts  yet 
to  be  born.  Nevertheless,  a  check  of  the  projections 
against  subsequent  census  counts  shows  important 
errors  in  the  older  cohorts,  too.  For  example,  projec¬ 
tions  of  the  population  6.'>  years  old  and  over  in  1%() 
prepared  in  lOS?  were  short  of  the  1960  census  couni 
by  6  percent.  The  principal  source  of  error  seems  to  be 
in  the  counts  both  in  the  1950  census,  upon  which  the 
projections  were  based,  and  in  the  1<>60  count,  which 
served  as  the  standard  to  which  the  projections  were 
compared.  The  explanation  of  these  errors  has  not  been 
determined. 


It  seems  then  that  errors  of  census  count  are  an  im¬ 
portant  source  of  error  in  demographic  projections  and 
also  complicate  the  measurement  of  the  success  of  the 
projections.  It  is  hoped  that  techniques  of  estimating 
net  census  undercount  will  become  sufficiently  reliable 
to  provide  acceptable  population  figures  as  a  base  for 
population  projections.  A  corrected  population  base  for 
the  projections  snd  corrected  population  distributions 
against  which  to  evaluate  them  should  make  a  consider¬ 
able  contribution  to  their  improvement. 

Projections  of  demographic  characteristics  may  often 
be  assigned  a  narrower  range  than  projections  of  the 
total  population.  An  analysis  of  the  participation  rates 
or  transition  rates  for  various  characteristics  often  in¬ 
dicates  that  they  may  be  afforded  considerable  confi¬ 
dence.  Hence,  the  uncertainty  about  births  is  the  prin¬ 
cipal  source  of  uncertainty  of  such  characteristics  as 
school  enrollment.  Where  birth  cohorts  are  not  in¬ 
volved,  such  as  projections  of  marriages  and  labor 
force  for  the  next  15  years,  the  projections  are  highly 
reliable. 

Alternative  sets  of  projections  are  often  calculated 
for  distributions  of  parent  population  among  States  or 
local  areas,  but  there  are  methodological  problems  in 
calculating  the  migration  component  in  such  a  way  as 
to  provide  reasonable  ranges  for  every  member  of  the 
di.slribution.  To  provide  maximum ■  and  minimums  of 
net  migration  for  each  area  would  require  computing 
as  manv  sets  of  projections  as  there  are  areas.  Methods 
of  obtaining  alternative  series  of  projections  bv  vary¬ 
ing  the  migration  component  were  discussed  above. 
Proje<iions  of  births  and  deaths  are  not  a  problem  in 
(his  respect,  since  the  alternative  assumptions  for  the 
parent  piqiulafion  can  be  used  without  changing  the 
basic  procedure  outlined  earlier. 
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DEVELOPMENT  OP  ALTERNATIVE  METHODS 


Attempt!  are  made  from  time  to  time  to  find  more 
mathematical  methoda  for  making  population  projec* 
tipaa.  Hie  most  notable  achievement  was  the  develop¬ 
ment  of  Msbie  population  theory  by  Lotka  in  ibe  19!M’s. 
He  demonstrated  that  a  schedule  of  age-specific  birth 
and  death  rates  applied  to  any  population  wiU  eventu- 
aUy  generate  a  predictable  age  distribution  and  rate  of 
popidation  growth  that  are  independent  of  the  initial 
age  distribution.  Lately,  Coale  and  Demeny  have  ap¬ 
plied  stable  population  theory  to  making  projections 
for  high  fertility  countries. 

More  recently,  attempts  have  been  made  to  apply 
matrix  algeb.a  and  Markov  chains  to  population  pro¬ 
jections.  The  cohort-component  method,  which  defines 
population  in  terms  of  a  population  distribution  and 

Mr.  Friedman.  There  is  an  old  adage  that  I  always 
felt  to  be  true;  The  rich  get  richer  and  the  poor  get 
children — is  there  any  validity  to  that? 

Mr.  Irwin.  The  lower  income  groups  do  have  larger 
families.  In  the  rural  South,  for  example,  the  nonwhite 
family,  both  white  and  nonwhite  families  the  per  capita 
income  is  very  low  with  high  birth  rates.  I  don’t  think 
the  situation  has  gotten  far  enough  to  see  whether  the 
same  conditions  that  prevailed  in  the  thirties,  when  the 
people  who  could  afford  to  raise  children  did  not  have 
them,  is  coming  true  now.  During  che  high  fertility  of 
the  fifties,  it  wasn’t  so  true.  Both  rich  and  poor  were 
having  children,  about  comparable-sized  families. 

Mr.  Cetron.  I'va  got  a  question.  It  seems  that  you’re 
controlling  and  comnensating.  If  your  figures  don’t 
come  out  the  way  you  want,  you  bring  in  more  migrants 
to  make  them  come  out  right.  You  are  actually  taking 
in  migrants  to  compensate  for  the  lack  of  validity  cf 
your  sample. 

Mr.  Irwin.  No,  we  are  not  that  chicancrous.  We’d 
probably  have  done  something  ■vhen  we  made  the  basic 
assumptions.  Just  before  the  immigration  started — 
that’s  happened  in  the  last  year,  and  in  that  blue  book 
we  make  it  very  simple.  1  must  admit  we  make  a  very 
simple  assumption  about  immigration,  the  national  pro¬ 
jection.  The  blue  b«rok  you  have,  286,  was  about 
800,000  a  year — it  had  been  running  around  that — and 
now  in  the  new  ones,  which  are  going  to  tome  out  and 
are  referenced  on  page  1 — we  are  assuming  400,000  a 
year.  The  latest  computation  is  over  400,000.  But  we 
didn’t  do  it  in  order  to  come  out  right,  however,  we 
would  like  it  to  come  out  between  B  and  C. 

Mr.  CmiON.  Can  you  modify  that  figure  with  figures 
going  out-  the  number  of  human  beings  on  the  conti- 


matrixea  of  transition  probabilities,  suggests  that  ma¬ 
trix  algebra  would  be  a  useful  tool.  However,  the  work 
so  far  suggests  that  it  can  add  little  to  what  is  already 
known  from  the  cohort-component  method  and  from 
stable  population  theory. 

The  methods  employed  by  economists  and  research¬ 
ers  in  other  fields  seem  to  have  little  application  to 
demography.  We  find  demographers  making  little  use 
of  regression  analysis,  simultaneous  equations  and  the 
like,  so  common  in  economic  projections.  Whether  the 
uniqueness  of  technique  is  truly  a  reflection  of  the 
nature  of  demographic  material  or  whether  it  ony  re¬ 
flects  the  parochial  view  of  demographers  might  well 
be  a  topic  to  discuss  at  this  conference. 

nent?  There  are  so  many  Americans  on  passport  who 
are  out  of  the  country  all  the  time. 

Mr.  Irwin.  We  do  take  this  into  consideration  as 
one  of  the  components.  We  have  about  five  categories 
of  net  immigration.  One  is  citizen  arrival  and  de¬ 
parture,  so  we  try  to  take  this  into  account,  and  we 
take  into  account  dependents  of  American  seivice  mer* 
coming  back  from  overseas,  for  example.  There  are  a 
number  of  places  where  statistics  are  not  perfect,  espe¬ 
cially  by  age  and  sex.  We  have  serious  problems.  Fur¬ 
thermore,  the  Bureau  of  fmmigration  in  1957  stopped 
giving  us  a  count  of  alien  emigrants — nut  alien  immi¬ 
grants,  but  emigrants  leaving  the  United  States  with 
the  announced  and  stated  intention  of  staying  away 
permanently.  We  do  try  to  take  these  into  account.  The 
Armed  Forces  abroad  are  also  taken  into  consideration 
— we  have  ■  million  men  overseas. 

Dr.  Fuu.er.  On  the  curves,  is  there  any  showing  of 
that  final  net? 

Mr.  Irwin.  No.  Once  a  year  we  have  a  report  on  the 
comp^ments  of  change  which  shows  some  of  the  varia¬ 
tions  from  year  to  year.  This  is  net  immigration  as  a 
rate  down  at  the  bottom.  This  is  l')51  and  that’s  the 
absolute  figure  of  about  a  year.  This  would  be 

my  basis  for  the  statement  that  it  is  relatively  constant. 
But  it  has  gone  up  as  a  result  of  changes  in  the  law. 

Colonel  Vauchn.  Not  too  long  ago  in  the  Washington 
Post  and  other  papers,  there  were  stories  about  the  .3 
million  to  .5  million  Americans  that  weren’t  counted  in 
the  last  cen$u.s.  What  is  that  going  to  d:>  to  your  chart? 

Mr.  Irwin.  When  you  read  my  paper  there  is  some¬ 
thing  pertaining  to  having  a  paper  reviewed  so  that 
every  iittir  jHiint  gets  in.  I  made  the  statement  th.st 
these  proje<-tions  of  the  population  that  are  alive  on 
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today’s  date  are  fairly  accurate.  We  found  that  projec¬ 
tions  made  during  the  fifties  when  compared  with  the 
1960  census,  for  the  older  age  groups,  were  not  good. 
Looking  further,  you  find  the  trouble  is  the  population 
is  undercounted  and  furthermore,  is  differentially  un¬ 
dercounted  by  age.  There  are  some  very  striking  things 
in  the  table — the  base  population  is  phony.  One  is  5 
percent  low,  another  is  only  2  percent  low.  These  kids 
are  really  good.  They  are  in  school,  mama  is  home 
taking  care  of  the  family.  We  do  good  with  those  kids, 
5-14.  Things  really  start  going  to  pot  when  we  get  in 
the  15-19.  For  white  males,  20-24,  the  undercount  rate 
is  about  7  percent  and  for  nonwhite  males,  20-24,  is 
20  percent.  We  misrod  one  out  of  five  of  every  nonwhite 
made  20-24.  Obviously  that’s  going  to  throw  things  off 
if  you  use  as  your  criterion  that  your  population  pro¬ 
jection  should  agree  with  the  u,  r.t  census.  I  think  the 
next  major  step  is  to  build  in,  and  work  with  the 
corrected  population.  In  other  woiJa,  first  we  correct 
the  population  for  undercount  and  we  now  have  an 
estimate  of  the  true  population,  just  an  estimate.  I  won’t 
use  the  words  “true  population.”  It  is  corrected.  Then 
carry  it  forward  and  either  state  our  projection  in  terms 
of  the  corrected  p<-)pulation,  which  would  require  a 
resolution  on  the  part  of  everybody  who  used  census 
statistics,  so  it  is  a  major  step.  Furthermore,  we  are 
not  that  sure  about  the  exact  undercount  by  age,  how¬ 
ever,  we  know  there  is  a  big  differential  in  undercount. 
We  know  some  of  the  percentages  are  big  but  we  don’t 
know  exactly  how  big. 

Dr.  Linstone.  Can’t  you  use  birth  and  death  data  to 
correct  population  projections  and  keep  them  more 
current? 

Mr,  Irwin.  Registered  births  with  the  native  bom 
population  enumerated  could  lot  be  done  until  about 
1940,  because  in  the  United  States  birth  registration 
was  not  even  nominally  complete  until  19.3.3  when  ihe 
State  of  Texas  entered  the  birth  re.gistration  area,  so 
until  that  time  there  were  no  printed  statistics  on  regis¬ 
tration  of  births  in  Texas  and  a  few  other  States.  Then 
in  1940  a  test  was  conducted  to  find  the  percentage  of 
under-registration  of  births.  From  this  we  take  the 
births  that  were  registered  when  the  area  was  finally 
complete,  correct  them  for  underregistration:  Apply 
mortality  rates;  compare  them  with  the  population 
enumeiated  in  1960  by  age;  by  cohort,  then  comparing 
it  with  births  during  a  stated  period  of  years:  They 
don’t  come  out. 

Dr,  Johnston.  I  wanted  to  make  comment  relating 
to  undercount,  too.  We  are  in  the  process  of  writing  an 
analysis  of  the  implicstion  of  the  undcrcount  for  labor 
statistics — that  is.  manpower  and  labor  force  atatiitics 


— in  which  we  will  do  something  to  the  review  and  ap¬ 
proval  of  the  Census  Bureau.  The  idea  I  want  to  get 
across  is  you  can  be  worried  about  the  effect  on  the 
levels  when  you  have  missed  5  million  people,  or  as  one 
man  put  if,  “You  missed  the  population  of  Denmark,  ’ 
or  something  like  that.  The  rates  may  not  be  affected 
significantly  because  we  don’t  have  too  much  evidence 
to  indicate  that  *he  people  missed  are  that  different 
from  the  people  who  are  caught,  at  Ic  ^t  in  the  same 
area— people  missed  are  not  likely  to  be  all  different 
from  the  people  caught,  we  are  interested  in  rates  of 
unemployment  on  rates  of  .school  attendance,  they  may 
not  be  that  far  off  even  though  the  levels  aren’t  there. 
This  is  always  subject  to  some  doubt.  ,\s  long  as  you 
have  missed  these  people,  you  really  don’t  know  what 
they  are  like. 

Mr.  Irwin  I  really  didn’t  answer  your  question. 
Colonel  Vaughn.  I’irst  of  all,  there  is  some  evidence  to 
show  with  respect  to  fertility  and,  for  example,  the 
difference  between  nonwhite  fertility  rates  as  printed 
and  white  fertility  rates.  There  is  a  differential  under¬ 
count  which  tends  to  bring  them  together.  But  it  is  not 
too  much  after  vou  do  it  with  the  best  figures  available, 
the  rates  and  the  trends  are  not  seriously  contradicted 
and  the  point  Denis  made  is  you  arc  not  quite  sure 
everybody  you  missed  had  a  different  proportion  of  the 
characteristics  you  are  talking  about.  Everybody  aa- 
sumes  that  all  the  inis.scd  peoply  are  unemployed;  that’s 
not  necessarily  true.  They  may  be  people  who  work, 
have  no  children  and  are  away  from  home.  Fnen  I 
might  defend  the  Census  Bureau  when  the  field  division 
is  trying  to  count  all  these  peogje.  We  have  a  lot  of 
ideas  about  how  to  do  it,*yhich  are  usually  expensive. 
Between  50-6(1  we  had.  this  same  constant  amount  r-f 
undcrcount,  alino.st  exactly.  We  mi.ssed  5  million  people 
in  1950  and  wc  mi.ssed  .5  million  in  l‘)60.  We  would 
have  had  about  .30  million  more  pe<iple,  *■>  the  rate 
went  down. 

Dr.  Dalkey.  If  you  look  at  the  dirtrihulion  of  ages 
in  the  almanac,  it  has  some  suspicious  lumps.  For 
example,  a  lump  at  the  age  of  59  and  another  lur.ip  in 
what  looks  like  it  might  be  preferred  ag«».  1  don’t  know 
whether  thu  is  due  to  misreporting. 

Mr.  iRVt^.  It  is  flue  to  misreporting  of  1900  as  the 
year  of  birA. 

Dr.  t>id  ‘o  ••Ite  this  info  account? 

Mr.  Irwin,  In  1960  the  C««sus  Bureau  had  a  dedi¬ 
cated  staff  so  they  said,  “We  are  not  going  to  ask  just 
how  old  were  vou  at  your  last  birthday,"  which  is  what 
I  will  answer  if  you  ask  me  how  old  I  am.  bid  thev  are 
going  to  ask  what  year  was  he  honi  in.  This  wss  done 
in  1960  because  when  you  isk  evv7l>ody  how  old  were 
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you  at  your  last  birthday,  there  is  a  distinct  preference 
for  ages,  like  45,  50,  60,  65,  and  so  on.  This  is  called 
age-peaking.  In  1960,  we  asked,  “What  year  were  you 
bom,”  and  this  caused  an  awful  lot  of  finger  counting 
at  the  door.  People  would  say,  “Let’s  see.  I’m  18,”  and 
so  forth.  However,  it  did  have  the  effect  of  eliminating 
the  age  preference  for  10,  20,  30,  40  years  and  the  only 
ones  marked  were  people  born  in  1900.  They  came  out 
as  59,  but  there  wasn’t  very  much  preference  for 
1910,  there  was  a  change  to  the  year  of  birth  which 
added  a  distinct  improvement  on  agency  tables  from 
the  point  of  view  of  peaking.  I  just  came  from  a  pretest 
of  the  1970  census  in  Philadelphia.  We  are  very  con¬ 
cerned  about  the  undercount  in  the  Negro  ghetto. 
There’s  no  question  about  it;  it  is  high.  That’s  where 
we  are  missing  males.  I  should  mention  that  the  compu¬ 
tations  of  undercount  give  no  indication  of  the  geo¬ 
graphical  location.  The  whole  thing  is  predicated  on  a 
closed  population.  You’ve  got  births  in  the  United 
States;  you’ve  got  native  born  people  and  yon  take 
account  of  the  people  abroad,  citizens  abroad.  Dr. 
Fuller,  end  you’ve  got  a  closed  population  but  that  s  all 
you  know.  You  don’t  know  whether  it  was  in  Harlem, 
in  Mississippi,  but  it  is  believed  to  be  in  the  ghetto. 

The  point  of  year  of  births,  I  observed  several  inter¬ 
views  with  people  with  low  education  and  our  schedule 
was  just  too  complicated.  It  is  going  to  be  tough.  The 
year  of  birth  requires  a  set  of  figures.  If  we  said,  “how 
old  are  you,”  that  part  of  the  question  would  be  simpler 
and  we  would  get  the  resc'  the  rest  of  the  infor¬ 
mation. 

Dr.  Fulled.  Since  1960  haven't  there  actually  been 
less  babies  each  year  in  the  United  States? 

Mr.  Irwin.  Wasn’t  1%3  the  highest  year  for  births? 

Dr.  Fuller.  We  actually  had  less  babies  and  I  think 
this  is  interesting  in  view  of  the  fact  that  we  would  have 
more  fertile  females  each  of  those  years,  would  we  not? 

Mr.  Irwin,  That’s  right. 

Dr.  Fuller.  So  we  would  have  higher  potential. 

Mr.  Irwin,  We  are  maintaining  that  the  present  drop 
in  the  volume  of  births  could  be  caused  by  a  change  of 
timing  and  age  of  marriage  and  does  not  necessarily 
reflect  a  decision  on  the  part  of  women  tc.  have  smaller 
families. 

Dr.  Fuller.  The  customs  of  people  in  America,  in 
the  last  decade  during  this  period  since  World  War  ll, 
has  changed — the  conduct  of  young  people.  Tlie  older 
people  not  exercising  their  will  over  the  young  peoples’ 
lives.  'The  young  people  have  had  much  more  oppor¬ 
tunity,  and  they’ve  lived  around  together  and  had 
more  opportunities  to  make  babies  than  they  ever  had 


before.  I  think  this  is  quite  a  significant  figure  and 
trend. 

Mr.  Monahan.  Why  the  drop  in  the  birth  rate  be¬ 
tween  1925  and  1929?  You  say  there  is  a  drop  of  the 
fertility  rate. 

Mr.  Irwin.  The  fertility  rate  dropped  all  through 
the  twenties,  I  believe.  The  best  discussion  I  ever  heard 
was  by  a  fellow  named  Fred  Chino  in  the  State  College. 
He  gave  the  best  analysis  and  I  have  never  heard  any¬ 
body  refute  it.  He  said,  “During  the  twenties,  first  of 
all,  the  women  having  babies  came  from  large  families 
and  their  mothers  were  generally  overworked,  but  more 
than  that,  women  had  just  achieved  equality.  They  had 
gotten  the  vote  in  1919,  so  this  was  combined  with  the 
feeling  on  the  part  of  women  that  having  babies  was 
not  the  only  aim  of  life  and  it  was  very  popular  to  think 
about  having  a  career.  So  a  woman  was  a  woman;  she 
had  a  baby  but  that  was  the  end  of  her  responsibility. 
She  was  going  to  go  out  and  make  her  way  in  ihe 
world.  As  this  period  continued,  this  attitude  was  the 
dominant  one.  Then  in  the  thirties  was  an  artificial 
financial  restraint,  so  this  deepened  and  then  the  war 
came  along  with  the  physical  restrictions  on  having 
children  and  all  of  these  things  let  loose  in  1946.  That’s 
my  answer. 

Major  Martino.  I  would  like  to  address  what  I  think 
was  your  original  question — the  methods  you  use  and 
so  on  rather  than  the  content  of  population  studies. 
You  put  up  a  straw  man  that  your  techniques  are 
unique  to  demography.  I  am  sure  you  don’t  believe  it 
and  I  believe  it  even  less. 

Mr.  Irwin.  I  sort  of  believe  it  but  I  am  ready  to  be 
shown. 

Major  Martino.  1  would  like  to  start — at  least,  try 
to  direct  the  discussion  in  that  direction  by  making 
some  assertions  of  my  own.  First,  any  forecasting  must 
of  necessity  be  based  on  data  from  the  pa.st.  That  is, 
there  is  no  other  way  to  make  a  forecast  except  to 
ignore  data  completely  and  just  gues.  If  you  attempt  to 
forecast  rationally,  you  must  start  with  what  you  al¬ 
ready  know  about  the  past,  to  this  extent  your  methods 
must  be  somewhat  similar  to  everybody  else’s.  Next,  I 
would  like  to  point  out  that  what  you  have  called  the 
component  equation  has  an  analog  in  physics.  This 
very  same  situation  is  known  as  the  equation  of  con¬ 
tinuity.  The  amount  of  material  or  whatever  you  have 
inside  some  boundary  is  that  derived  from  the  sources 
less  that  lost  to  the  sinks,  plus  the  ner.t  influx  across  the 
boundary.  There  again  is  something  in  your  model 
which  is  a';  least  paralleled  by  something  in  somebody 
else’s  model.  This  is  to  reinforce  my  contention  that 
your  methods  are  not  necessarily  unique. 
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Mr.  Irwin.  Very  interesting.  What  about  this  ques¬ 
tion?  What  we  are  really  interested  in,  is  whether  you 
people  can  see  parallels  that  would  help  us  in  this 
question  of  period  fertility.  By  that  I  mean  the  rate  at 
which  all  the  women  alive  at  one  time  are  having  babies, 
and  on  the  other  hand  the  idea  of  following  a  women 
through  time  until  she  is  through  having  babies.  Can 
we  tie  these  together  in  some  way? 

Dr.  Slafkosky.  I  question  your  definition  of  fertil¬ 
ity,  because  you  are  applying  it  only  to  women  who 
have  proved  their  fertility. 

Mr.  Irwin.  Very  true.  The  state  of  our  art  is  a  crude 
way  to  get  it.  In  my  discussion  of  migration  and  sta¬ 
tistics,  we  are  quoting  a  rate  of  having  babies  by  .all 
women  whether  married  or  not  and  whether  fecund  or 
not.  We  should  quote  rates  of  married  women.  I  am 
realizing  the  dangers  of  having  a  verbal  discussion. 
I  left  out  one  whole  technique  of  projecting  births 
which  is  the  parity  progression  method.  The  completed 
fertility  technique  of  projecting  births  is,  we  think 
best,  in  the  longrun,  but  in  the  shortrun  we'  are  not 
doing  so  well.  It  is  very  hard  to  get  long  range  estimates 
of  fertility  of  women  to  come  out  on  the  short  run.  So 
when  we  put  the  computations  together  we  very'  typi¬ 
cally  get  nonsense  results  in  the  first  few  years.  It  is  a 
big  problem.  There  is  another  way  of  doing  it  but 
unfortunately  there  is  not  enough  data,  even  at  the 
national  level,  and  this  is  to  go  back  to  a  cohort  pro¬ 
gression  of  women  in  effect  and  use  the  cohort  ap¬ 
proach  with  the  women.  Take  ail  the  women  of  a  given 
age  and  assign  them  a  parity:  How  many  babies  have 
they  already  had,  then  you  say  the  chance  that  these 
women  of  different  parities  will  have  a  baby  during 
the  next  time  period  is  such  and  such.  Then  you  say, 
“OK.  This  is  all  women,  aged  20-24,  who  have  had 
one  baby,  two  babies,  three  babies,  four  and  more,  you 
assign  different  probabilities  and  multiply  these  out.” 
There  has  been  some  experimentation  on  this  and  the 
tables  are  hard  to  come  by.  Particularly  the  tables 
showing  parity  by  length  of  intervals  since  the  last 
baby,  which  is  an  extremely  vital  factor,  because  a 
woman  who  had  one  child  5  years  ago  is  less  likely  to 
have  another  child  than  a  woman  who  has  had  one 
baby  12  months  ago.  This  is  called  the  parity  progres¬ 
sion  method.  You  can  read  our  comments  in  the  paper 
at  your  leisure.  We  would  like  to  use  parity  progression 
approach  in  the  shortrun  and  join  it  with  the  cohort 
fertility  in  the  longrun  but  it  is  quite  a  trick  in  compu¬ 
tation. 

Dr.  Slafkoskv.  May  I  suggest  that — in  some  way 
you  probably  have  a  more  difficult  problem  than  most 
of  us  and  I’m  not  sure  we  are  ever  going  to  get  the  kind 


of  conclusions  you  want.  You  are  dealing  with  a  subject 
matter  that  has  many  variables  and  the  kinds  of  influ¬ 
ences  you  gave  Mr.  Monahan  are  the  overly  large 
problems.  I  have  a  feeling  there  are  a  lot  of  medium- 
size  and  smaller  ones  that  work  with  individuals,  fami¬ 
lies,  groups,  and  sectors  that  you  will  never  get  your 
finger  on  and  these  are  the  ones  that  are  always  giving 
you  trouble.  Even  if  you  talk  about  a  parity  for  a 
cohort  under  similar  conditions,  carry  it  through  and 
always  look  at  it  in  that  way:  You  are  not  going  to  be 
set  on  any  firm  basis  to  make  your  prognostications. 

Mr.  Irwin.  We  could  further  disaggregate.  At  the 
present  time  it  is  a  great  problem  using  the  parity 
progression  because  we  are  not  working  up  good  tables 
showing  parity  as  it  passes  through  time.  We  are  going 
to  want  to  look  at  some  trends  before  we  project  it  into 
the  future.  As  a  general  observation  which  I  might 
make,  going  back  to  the  undercount.  The  basic  quality 
of  population  in  statistics  is  not  that  good.  They  are 
never  going  to  get  it.  People  don’t  report  well.  You  miss 
too  many  people  and  we  get  a  lot  of  resistance.  We’ve 
got  29  housing  questions  and  about  30  population 
auestions,  that  will  go  to  20  percent  of  the  people;  and 
plenty  of  studies  to  show  that  the  answers  jiggle.  For 
example,  mobility  and  migration,  we  asked  them  where 
they  lived  5  years  ago  and  only  90  percent  of  the  people 
who  actually  moved  interstate  on  the  basis  of  a  reinter¬ 
view  study  marked  it  that  way  on  the  questionnaire. 
There  is  about  10  percent — we  arc  talking  about  big 
numbers.  I  feel  that  further  disaggregation  is  not  feasi¬ 
ble  at  this  time.  It  is  certainly  something  to  consider. 

Dr,  Chacko.  Would  you  be  able  to  prove  your  allo¬ 
cation  method  by  the  apportionment  of  migration  to  the 
50  States  which  is  2  percent  on  the  basis  of  the  null 
hypothesis?  It  seems  to  me  you  mentioned  two  very 
important  factors,  those  of  the  military  establishments 
and  universities,  the  actual  number  of  students  and 
military  personnel  are  nearly  constant,  invariant.  So, 
by  ascertaining  these,  couldn’t  we  say  tl.at  these  popu¬ 
lations  had  to  come  from  some  place  and  they  arrive, 
and  use  that  as  a  refinement  of  the  allocation  of  the 
migrant  population? 

Mr.  Irwin.  We  do  make  some  allocationc  actually 
based  on  tabulation.  In  making  our  population  esti¬ 
mates,  I  have  talked  all  about  projections — the  popula¬ 
tion  estimates  for  areas  generally  use  some  of  the  same 
techniques;  and  in  the  case  of  making  our  national 
estimates  by  age,  sex,  and  color,  we  start  from  the  base 
population,  add  registered  births,  subtract  registered 
deaths  by  age,  add  immigration  as  computed  by  our 
various  methods,  and  in  the  process  we  have  the  Armed 
Forces.  We  get  statistics  from  the  Armed  Forces  which 
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m  bM»4  on  maples  that  are  different  for  each  brandi 
of  the  Mrrice,  but  we  use  thorn  as  one  unit  and  thi> 
shows  preservioe  residence  of  each  member  of  the 
Armed  Forces.  Then  we  have  to  allocate  them  to  their 
former  honm  or  assume  where  they  are  gouig  to  move, 
after  being  discharged.  They  are  all  estimates  and  are 
all  pretty  tricky.  For  example,  I  can  get  distribution 
of  Armed  Forces  by  branch  for  the  Nation,  but  I  don’t 
know  what  it  is  for  Alamogordo,  Randolph  Field,  or 
Fort  Banning.  I  know  a  group  of  young  people  are  at 
Fort  Benning  for  basic  training  but  that’s  all  I  know, 
so  when  you  get  down  to  a^  .  there  is  a  problem. 

Major  Martino.  I  would  like  to  return  again  to  my 
previous  thesis.  You  posed  a  sequence  of  models  of 
improved  nature,  the  first  being  a  net  migration  model, 
net  immigration,  where  you  used,  as  I  understood  it,  a 
proportionality  factor — the  net  immigration  being  pro¬ 
portional  to  what  was  already  there  and  you  have 
shown  that  this  had  an  inherent  bias  in  the  computa¬ 
tion.  This  is  a  problem  in  the  mathematics  whicn  has 
nothing  to  do  with  reality.  Next  you  gc  to  a  modified 
migration  stream  method  which  again  has  an  inherent 
difficulty  in  the  mathematics  because  it  assigns  a  con¬ 
stant  proportion  of  the  migration  pool  to  certain  States. 
This  is  hissed  against  the  rapidly  growing  States  and 
for  the  lesser  rapidly  growing  States  and  so  on.  I  am 
suggesting  in  effect  two  things:  You  haven’t  gone  far 
enough  in  this  trend  away  from  the  net  migration 
model  and,  barring  difficulties  of  gathering  the  data 
which  may  be  your  biggest  problem,  what  you  can 
move  to  next  is  to  recognize  that  this  thing  is  not  much 
different  from  a  Markov  Process.  You  actually  have  a 
so  by  SO  matrix  of  transition  probabilities  and  the 
probability  of  a  person  in  .State  A  going  to  State  B  is 
an  entry  in  each  cell.  The  matrix  is  not  triangular 
because  above  the  diagonal  you  have  from  A  to  B  and 
below  the  diagonal  you  have  from  B  to  A.  The  sum  of 
the  two,  the  net  sum,  must  be  the  net  probability  of 
going  from  A  to  B.  The  point,  however,  is  this  would 
eliminate  the  difficulties  both  with  the  net  migration 
model  and  with  this  peculiar  allocation  process  of  the 
migration  pool  model  in  that  it  creates  no  mathematical 
artifacts  in  your  modeling,  and  assuming  you  get  the 
data,  it  allows  you  to  go  directly  to  the  growth  rates 
of  each  individual  sta*e. 

Mr.  Irwin.  A  very  good  point  and  I  think  I  ought 
to  meet  it.  I’m  with  you,  but  I’ll  go  against  you  just 
for  the  record.  First,  I  like  the  assumption  that  might 
be  argued  on  a  philosophical  basis — I  find  the  Bureau 
feels  that  we  should  assume  a  convergence  toward  a 
national  unity  in  all  of  these  variables.  For  example, 
births:  We  don't  compute  births  separately  for  each 


State,  we  compute  a  ratio  between  the  fertility  rate  per 
.iitate  A  in  the  Nation  and  State  B  in  the  Nation,  then 
we  assume  those  ratios  are  going  to  be  1  at  a  point  SO 
years  in  the  future.  Hiis  illustrates  the  turn  of  mind. 
In  the  same  sense,  using  a  constant  set  of  proportions 
will,  in  a  period  of  time,  produce  zero  net  migration 
because  sooner  or  later  it  will  even  out.  We  like  it  and 
we  argue  that  the. differentials  which  exist  at  the  begin¬ 
ning  of  the  projection  period  will  tend  to  eliminate 
themselves  as  time  proceeds.  We  don’t  know  what  the 
differential  factors  are  going  to  be.  They  are  going  to 
enter  into  the  future,  sc  we  won’t  try  to  estimate  them 
— we  like  the  convergence.  If  I  don’t  like  the  con¬ 
vergence,  it  is  extremely  easy  to  use  this  model  and 
not  have  convergence.  All  I  have  to  do  is  compute 
inmigration  as  rates,  build  up  a  fictitious  inmigration 
pool  for  each  projection  period,  compute  the  propor¬ 
tions  to  the  larger  inmigration  pool  and  then  allocate, 
using  those  proportions.  The  large  States  will  gain,  too, 
and  I  can  assume  no  convergence.  The  basic  data  is 
important  because  it  is  hard  enough  to  come  up  with 
an  efltimr^e  about  the  impact  of  special  populations  on 
this  model — I’m  not  very  good  at  Markov  chains — ^but, 
it  seems  to  me,  it  would  be  rougher  to  get  into  a  place 
to  place  migration  stream  model  although  in  my  paper 
I  do  maintain  and  I  will  maintain  it  honestly  and  vigor¬ 
ously  that  we  should  explore  this,  it  is  obviously  a 
direction  that  research  should  take,  no  disagreement. 

Dr.  Linstone.  Do  you  take  into  account  the  techno¬ 
logical?  You  mentioned  the  economic  factor.  Bow  are 
technological  factors  like  the  oral  contraceptive 
business? 

Mr.  Irwin.  Speaking  of  employment  first,  we  are 
looking  at  other  people’s  studies  of  employment  and 
labor  force.  On  births,  there  is  a  lot  of  work  being  done 
on  fertility,  fertility  control,  the  impact  of  various 
methods  of  contraceptives  on  fertility.  Work  is  being 
done  ip  the  colleges  which  we  are  studying.  We  have 
not  explicitly  built  it  into  a  population  model,  we  are, 
however,  keeping  abreast  of  the  research  in  this  field. 
My  feeling  is  the  impact  of  the  oral  contraceptive  on 
the  birth  rate  at  the  present  tims  is  somewhat  over¬ 
rated.  The  fact  that  fertility  got  to  the  lowest  period 
ever  experienced  in  this  country  in  1933  without  one  is 
proof.  If  women  decide  they  don’t  want  children,  they 
managed  to  prevent  it  although  I  don’t  deny  the  factor 
and  obviously  it  is  a  factor.  Contraceptives  will  have 
an  impact  but  not  a  decisive  one. 

Dr.  Chacko.  Following  up  Major  Martino’s  remarks, 
your  last  statement  about  the  relationship  between 
economic  and  demographic  factors  brings  up  the  simi¬ 
larity  of  forecasting  techniques  that  you  use  in  demog- 
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raphy  and  elaewhere.  I  think  I  would  take  itrae  with 
Major  Martino  in  using  the  Markov  Process.  In  order 
to  use  the  Markov  Process,  you  need  a  transition  ma¬ 
trix  which  has  constant  probabilities.  The  assumption 
that  the  probabilities  are  constant  with  reference  to 
migration  is  too  restrictive.  I  suggest  you  look  at 
Leontiefs  Input-Output  Table.  It  would  be  a  50  x  Su 
table.  You  can  derive  the  technical  coefficients  for 
inmigration  into  California  from  other  States.  You  can 
apply  this  coefficient,  say  0.35  to  the  next  period  and 
see  how  close  it  is  to  the  actual  inmigration.  Because 
the  table  requires  that  arithmetically  the  inputs  should 
equal  the  outputs,  you  force  the  net  migration  to  be 
zero.  I  am  mentioning  this  as  one  technique,  I  didn’t 
hear  you  refer  to  any  logistic  curves.  Have  you  given 
up  using  these? 

Mr.  Irwin.  Well,  pretty  much.  Just  the  same  as  our 
low  projection  of  population  in  1946  wiped  out  the 
period  fertility  and  let  us  do  so  much  work  on  that 
same  technique;  in  the  early  1900’s  some  people  named 
Pearl  and  Reed  developed  a  complicated  curve.  This 
has  been  completely  dropped.  Again  there  and  then, 
the  fact  that  the  fertility  doesn’t  seem  to  line  up  with 
major  economic  shifts  is  another  thing  that  troubles  us. 
But  I  think  it  may  begin  again.  Who  knows? 

Dr.  Chacko.  In  light  of  what  Dr.  l.instone  has  stated, 
I  was  wondering  if  you  had  considered  the  use  of  the 
exponential  smoothing.  What  it  does  basically  is  take 
and  weight  the  data  differently,  giving  say  80  percent 
of  weight  to  the  last  data  and  maybe  20  percent  to  all 
the  previous  period  data.  You  would  get  a  continued 
extrapolation  which  takes  into  account  all  the  variance 
of  the  data.  Brov.  i  at  Arthur  D.  Little  developed  this 
program.  Its  advantage  is  that  if  you  have  sudden 
changes  such  as  the  one  mentioned — the  use  of  oral 
contraceptives — the  smoothing  program  will  reflect  the 
latest  and  drastic  changes.  I  have  used  this  successfully 
in  forecasting  the  growth  of  chemical  products  which 
are  quite  volatile,  starting  slow  and  zooming  up.  This 
has  been  found  to  be  useful.  Have  you  considered  that? 

Mr.  Irwin.  We  have  talked  about  it  and  we  have  a 
great  need  for  good  shortrun  projection.  I  want  to 
figure  this  summary  for  foreign  demography  in  the 
sense  that  I  want  a  publishable  figure  for  January  I, 
1968.  Well,  it’s  a  little  embarrassing  for  us.  We  have 
a  rather  crude  extrapolation  which  goes  into  the  census 
for  1970.  We  don’t  have  any  consistent  way  of  fore¬ 
casting,  say,  3  months  or  6  months  ahead.  We  do  feel 
we  should  do  that.  This  is  a  very  good  suggestion  and 
we  are  interested  in  this  line  of  reasoning. 

Major  Martino.  I  would  like  to  add  to  Mr.  Linslone’s 
comment  regarding  technological  impact.  Do  you  at¬ 


tempt  to  forecast  area  population  or  could  you  uae 
technological  impact,  in  such  things  as  replacement  of 
coal  ly  oil  as  an  energy  source  and  use  this  to  conclude 
that  the  population  of  West  Virginia  is  going  to  drop? 

Mr.  Irwin.  I  am  sure  the  Bureau  of  Census  would 
feel  that  is  a  function  of  other  analysts.  We  will  use  the 
research  that  other  people  do  both  public  and  private. 
I  would  nsk  about  the  idea  I  mentioned  in  the  model 
which  was  outlined.  In  other  words,  give  me  an  estimate 
of  employment:  I  am  going  to  let  other  people  decide 
how  you  do  it.  I’ve  got  enough  complexities  of  my 
own.  I  don’t  know  if  that  is  the  right  attitude  but  I  feel 
we  are  in  an  age  of  too  much  specialization.  The  feeling 
is  that  we  will  try  to  coordinate  our  efforts  with  other 
people  who  are  doing  that. 

Mr.  Cetron.  Going  back  to  Major  Martino’s  point 
again,  where  is  it  similar  to  the  forecasting  techniques 
that  are  being  utilized  now  and  I  think  what  we  are 
using  now  is  extrapolation  basically  with  ail  that  it 
entails.  If  everything  goes  or.  as  it  did  in  the  past,  it 
will  be  fine  in  the  future.  We  should  be  looking  at  this 
migration  figure  using  Dynamic  Forecasting  Tech¬ 
niques,  and  use  these  trends  for  efficiency  where  the 
weights  are  put  on.  fhat’s  what  Joe  and  Dr.  Chacko 
are  trying  to  get  at. 

Mr.  Irwin.  This  is  a  very  interesting  point  because 
I  have  discussed  it  with  various  people  in  the  Bureau 
and  one  thing  we  have  noticed:  Demographers  as  a 
general  group  feel  rather  guilty  about  the  fact  that  we 
are  just  projtcting  those  rates  out,  so  we  have  a  guilt 
complex;  but  the  interesting  fact  is  that  the  people  who 
are  forecasting  the  economy  are  usurlly  doing  the  same 
thing,  so  what  happens  is  when  you  equate  these 
things,  they  are  not  as  different  as  you  might  think. 

Mr.  Cetron.  Yes,  but  that  doesn’t  mean  two  wrongs 
make  a  right.  \6u  are  checking  your  standard — if  the 
standard  is  wrong,  then  they  are  both  incorrect, 

Mr.  Irwin.  You  are  right.  1  would  restate  by  saying, 
until  1  see  an  economic  model  is  going  to  do  it  better. 
I’m  saying,  “Look,  give  me  an  estimate  of  employment 
and  I  will  tie  my  projection  to  it  but  that  estimate  of 
employment  has  got  to  be  more  dynamically  oriented 
than  mine.” 

Mr.  Hacke.  Thi*  is  really  a  question  for  information. 
I  have  the  impression  and  it  may  be  simply  that  Tve 
been  brainwashed,  when  you  go  to  a  smaller  aggregate, 
neighborhood,  community  or  metropolis,  that  the  peo¬ 
ple  in  the  utilities  planning  do  a  very  worthwhile  job 
of  demographic  forecasting.  I  want  io  know  first,  do 
you  believe  this  and  second,  is  there  a  microstructure 
of  demographic  forecasting  for  people  who  move  where 
there  are  utilities  and  therefore  fulfilling  a  prophecy. 
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BIr.  bwm.  By  ntilitiM,  what  do  yon  mean? 

Mr.  Hacki.  Sewage,  water,  lit^ita. 

Mr.  lawiK.  I  think  the  buiklin,^  contractora  do  that, 
as  to  where  yon  pot  the  houses,  to  the  extent  that  deter¬ 
mines  where  the  people  go;  I  think  its  very  true.  With 
respect  to  local  projecting,  from  the  national  point  of 
view,  I  can  represent  both  positions  because  Pve  only 
been  with  the  Bureau  a  year.  I  worked  for  Oregon  and 
California  and  there  u  a  lot  of  feeling  on  the  part  of 
a  local  person  that  he  can  better  assess  the  individual 
potential  and  factors  uf  his  area.  I  do  fed  at  the  Fed¬ 
eral  levd  ore  great  danger  is,  you  lose  contact  vrith  the 
local  things.  For  example,  if  I  were  to  retain  the  same 
kin'*  of  contact  with  each  State,  nuking  State  projec¬ 
tions,  it  would  take  a  huge  travd  budget  which  I'd 


never  get  one  percent  of.  There  is  the  danger.  On  the 
other  hand,  there  is  a  kind  of  doubt  that  these  things 
are  not  mdependent;  when  you  come  at  it  from  the 
top — for  instance,  thL  gentknun  mentioned  finding  a 
way  to  make  a  good  economic  employment  projection, 
starting  with  an  input-output  projection  in  the  national 
picture  for  1970  or  197S,  manage  to  work  thic  down  to 
the  States  hy  a  good  allocation  system  and  then  plug 
this  into  niy  demographic  model  and  get  it  all  lined  up. 
On  the  average,  we  are  going  to  do  better  than  these 
areas  would  do  alone.  “On  the  average”  is  a  weasel 
word.  It  reminds  me  of  the  only  joke  I  know  about 
statistics.  A  statistician  is  a  man  who  when  he  comes 
in  the  room  and  finds  a  fellow  with  his  feet  in  the 
icebox  and  his  head  in  the  oven,  says,  “On  the  average, 
you  are  very  comfortable.” 


Dr.  North  has  applied  the  Delphi  procedure,  dwribed  earlier  by  Dr.  Dalkey,  to 
the  task  of  preparing  plans  for  his  company.  His  description  of  how  the  Delphi  pro¬ 
cedure  worked  in  a  practical  situation  should  be  of  interest  to  those  contemplating 
the  use  of  it. 

'  — Editor 
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TECHNOLOGICAL  FORECASTING  AND 
ITS  ROLE  IN  PLANrJING 

Harper  Q,  North,  RcMorch  and  Dovolopmont,  TkW  Inc, 


I  first  first  met  Joe  Martino  at  a  conference  at  Lake 
Fiacid,  N.  Y.,  on  May  23  sponsored  by  the  Industrial 
Management  Center  of  the  Harvard  business  school. 
Joe  invited  my  participation  in  this  conference,  and  I 
still  feel,  as  I  felt  at  that  time,  that  the  setting  of  coipo- 
rate  goals  in  any  company  and  the  establishment  of  a 
sound  research  and  development  program  constitutes  a 
chicken-and-egg  situation.  Today,  I  want  to  elaborate 
upon  that  statement  and  to  describe  a  program  in 
which  I — and  more  recently  my  peers  and  superiors — 
have  expressed  considerable  interest  and  to  which  my 
office  is  devoting  considerable  time.  I  hope  that  you 
will  forgive  any  flavor  of  advertising  in  the  event  that 
I  become  carried  away  and  mention  TRW.  Any  refer¬ 
ence  to  a  specific  company  living  or  dead — and  TRW 
is  certainly  living — will  be  in  the  interest  of  making 
real  an  exercise  “which  might  otherwise  be  classified  as 
pure  theory. 

Those  of  you  who  attended  the  Lake  Placid  confer¬ 
ence  may  sleep  through  the  first  half  of  my  talk,  as  I 
plan  to  start  from  the  beginning  of  our  technological 
planning,  if  only  to  use  some  of  the  excess  time  which 
Joe  Martint.-  has  allocated  to  me.  In  fact,  many  of  you 
will  recognize  our  exercise  as  a  variation  and  extension 
of  a  study  conducted  by  Major  Martino  and  his  group 
in  the  Office  of  Scientific  Reseaten.  Their  study  pre¬ 
dicted  events  in  politics,  economics,  and  the  military. 
Ours  is  confined  to  lecimology. 

Ijite  in  1965,  my  assistant,  Don  Pyke,  called  my 
attention  to  the  now  well-known  article  in  “News 
Front”  fir  April  of  that  vear  by  Olaf  Helmer  of  Rand 
and  Te«'  Gordon  of  Douglas  entitled,  “Probing  the 
Future.”  I  was  quickly  persuaded  tb.st  the  Delphi 
technique  could  readily  be  adapted  to  research  and 
development  planning,  but  persuading  even  an  imagi¬ 
native  president  and  chairman  of  TRW  of  the  value  of 
the  project  required  a  little  stratr.gy.  My  entre  w.-rs 
through  Dr.  Simon  Ramo,  longtime  friend  and  asso 
ciate  and  vice  chairman  of  TRW’s  board.  On  the  human 
side.  Si  Ramo  is  a  friend  hut  devil’s  advocate.  He  will 
pose  prob:'!g  questions  one  after  another  until  the 


proponent  is  ready  to  say,  Forget  it!  but  just  short  of 
that,  he  will  say.  Let’s  try  it.  So,  tentative  plans  to 
probe  TRW’s  future  were  presented  to  about  200  of 
our  key  technical  people  at  dinner  meetings  in  Los 
Angeles  and  Qeveland.  The  reaponae  before  Los  An¬ 
geles  engineers  of  the  systems  group  was  encouraging 
but  reserved.  In  Gtveland,  the  experienced  and  con¬ 
servative  automotive  engineera  were  merely  reserved, 
but  we  had  enough  to  start  on. 

I  asked  the  executive  vice  presidents  of  our  four 
TRW  groups:  Automotive,  electronics,  aircraft  equip¬ 
ment.  and  systems  for  five  of  their  moat  imaginative 
but  practical  engineers  and  scientists  to  serve  so  pand- 
dsts  for  an  exercise  in  the  Delphi  technicpie  of  predict¬ 
ing  the  technological  future  with  whidi  TRW  would 
have  to  cope  in  the  20  years  ahead. 

During  the  short  initial  meeting  with  each  of  the 
group  of  experts  (whose  spare  time  had  been  prom¬ 
ised),  Don  Pyke  and  I  established  the  following 
ground  rules: 

1.  Each  man  was  asked  to  list — independently — 
those  technical  events  which  he  felt  might  take  place 
between  1966  and  1985  and  which  would  have  a  aub- 
stantial  impact  upon  the  current  or  potential  product 
lines  of  his  group. 

2.  Each  panelist  was  then  asked  to  indicate,  for  each 
event,  a  date  by  which  he,feh  there  was  a  50-percent 
probability  that  the  event  would  have  occurred. 

3.  He  was  invited  to  consult  fredy  with  anyone 
throughout  the  company,  but  his  list  of  events  would 
contain  only  those  which  he  thought  would  occur. 

4.  Although  we  reengnixed  at  that  time  that  ^hc 
environment  of  *he  ’uturr  would  have  a  major  impact 
upon  the  directions  which  technology  would  take,  we 
had  no  accepted  or  official  picture  of  that  environment 
to  offer.  We  did  supply  each  panel  member  with  what 
scanty  data  we  had  at  hand,  including  a  McGraw-Hill 
economic  forecB*\  the  resuhs  of  the  Helmer-Gordon 
study,  a  ‘Time”  magazine  essay  on  the  future,  and  a 
“U.S.  News  and  World  Report"  article  blueprinting 
the  future  United  States.  Beyond  this,  each  person  was 
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idyited  to  nuke  hie  own  easumptiona  concerning  the 
future  environi\.ent,  excluding  only  all-out  nuclear  war. 

About  a  month  waa  allowed  for  the  return  of  predic- 
tiona  from  all  pandiats.  Event  descriptions  were  edited, 
combined,  and  grouped  according  ti  iechnologies,  and 
a  croas-indexe^  Srst  draft  was  returned  to  each  panelist 
with  the  requi^  that  he  edit  freely  in  areas  of  his  own 
expertise  (1)  defining  events  more  precisely,  (2) 
deleting,  adding  or  otherwise  modifying  event  de¬ 
scriptions,  and  (3)  indicating  his  own  estimated  date 
of  occurrence  in  case  of  disagreement  with  the  dates 
predicted.  At  this  point,  we  should  have  known  better 
than  to  furnish  dates,  offering  the  opportunity  to  con¬ 
cur  or  disa",fee.  Human  nature  being  what  it  is  resulted 
in  concurrmce  much  more  often  than  disagreement, 
which  requires  careful  deliberation. 

Comments  of  the  panelists  were  carefully  considered, 
combined  where  appropriate,  reworded  as  necessary, 
and  listed  under  15  categories  chosen  empirically  to 
encompass  all  events.  Events  which  could  be  included 
in  more  than  one  category  were  placed  in  that  category 
through  which  the  greatest  impact  upon  TRW’s  future 
might  be  feh. 

The  result  of  our  exercise  was  a  50-page  booklet 
covering  401  evenU  and  entitled,  “A  Probe  of  TRW’s 
Future/The  Next  20  Years.” 

Figure  IV-l  lists  the  15  s,reas  under  which  events 
were  categorised.  These  relate  to  techi.ologies,  sub¬ 
systems,  and  systems  of  interest  lo  TRW. 

By  way  of  illustration,  I  am  going  to  quickly  cover 
figures  which  I  presented  to  YEW’S  1966  management 
conference.  Figure  IV— 2  lists  6  of  the  everts  which  we 
felt  would  have  a  significant  impact  upon  our  automo¬ 
tive  group,  which  is  the  principal  supplier  of  valves  for 
the  United  States,  and  furnishes  chassis  parts  for  auto¬ 
mobile  manufacturers.  Figure  IV-3,  IV-4,  and  IV-5 
show  the  results  for  the  other  six  groups.  The  lines  indi¬ 
cate  the  spread  of  dates  estimated  by  all  panelists,  and 
the  median  estimate  is  indicated  by  a  dot.  Where  there 
it  no  spread,  artificial  concurrence  due  to  laxiness  on 
the  part  of  our  pa.iel  members  can  probably  b»‘ 
assumed. 

In  figure  IV— 3  are  some  of  the  predicted  events 
which  are  certain  to  have  an  impact  upon  divisions  in 
cur  electronics  group.  You  will  note  that  the  nature 
of  the  events  predicted  is  compatible  with  the  concen¬ 
tration  bt  this  group  upon  components  rather  than  upon 
systems. 

Figure  rV-4  iliustrstes  selected  e\ents  which  will 
have  a  substantial  impact  upon  our  equipment  group. 
This  group,  os  you  will  recall,  produces  equipment 


primarily  for  the  aircraft,  but  also  for  the  aerospace 
and  defense  industries. 

Figure  lV-5  lists  selected  events  which  will  have  an 
impact  upon  our  systems  group.  As  you  may  know,  this 
group  cut  its  teeth  on  ballistic  missile  systems  ard  then 
turned  to  spacecraft,  participating  in  90  percent  of 
U.S.  space  launches.  At  its  inception  14  years  ago,  sales 
were  100  percent  software.  They  are  now  two-thirds 
hardware.  Recent  software  activities  (part  of  the  one- 
third  of  sales  re  maining)  include  the  study  and  design 
of  such  civil  systems  as  hospital  systems,  high  speed 
ground  transportation  systems,  economic  land-use 
studies,  etc. 

Upon  completion  of  Probe  I,  the  exercise  had  caused 
27  of  TRW’s  most  creative  engineers  and  scientists  - 
and  the  additional  technical  people  with  whom  they 
had  talked — to  extend  their  imaginations  beyond  the 
specific  demanding  tasks  of  the  moment.  Not  much 
more  had  been  accomplished. 

In  July  of  last  year,  I  presented  the  results  of  cur 
study  at  TRW’s  annual  management  conference  in 
Vermont,  using  the  figures  which  you  have  already 
seen.  The  response  was  encouraging — 12  copies  of  the 
probe,  which  I  had  left  on  a  table  near  the  door,  dis¬ 
appeared  immediately.  Since  that  time,  more  than  400 
copies  of  our  first  probe  have  been  distributed  by 
request  throughout  the  corporation.  A  sanitized  version 
of  the  probe  was  prepared  in  response  to  requests  from 
our  public  relations  group  and  from  people  in  finance 
as  background  material  for  discussions. 

The  climax  in  publicity  came  in  an  article  which 
appeared  in  “Business  Week”  for  May,  1967.  In  that 
article,  we  read  in  print  for  the  first  time  that  tech¬ 
nological  forecasting  was  a  part  of  TRW’s  long-range 
planning  effort.  The  probe  hrd  been  launched  as  an 
official  document  for  TRW. 

Don  Pyke  and  I  knew  of  the  shortcomings  of  the 
initial  experiment; 

1.  There  had  been  no  forma!  use  made  of  the  probe 
in  corporate  or  group  planning. 

2.  Some  very  important  areas  were  sparsely  cov¬ 
ered.  These  included  microelectronics  and  automotive 
developments. 

3.  There  were  many  inconsistencies.  Events  were 
predicted  at  dates  which  preceded  other  events  that 
were  necessary  for  their  completion. 

An  exercise  is  currently  in  process  which  is  designed 
lo  correct  errors  made  in  the  first  experiment.  'Ihe 
results  will  be  entitled  “TRW’s  Probe  il — A  Guide  to 
Growth  Through  Technology.”  Several  improvements 
have  been  made  in  ll  >■  ihod  used  for  this  second 
edition  which  will  t  more  than  fine  tuning  on 
Probe  I. 
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PROBE  OF  TRW'S  FUTURE  CLASS  IFiCATiON  OF  EVENTS 
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MATERIALS  PROCESSING  &  HANDLING  PLUS  AUTOMATION 


'85  BD'CMO 


f 


9 


t 


Q 

UJ 

:d  . 


I 


o 


d_, 

5  > 

UJ  ^ 

oo  < 
_j  X 

*  Z 

a2 


»/> 

p£ 


a 

1— 

X 

o 


?< 

X  <J 

u  ~ 

UJ  ^ 

f—  <1, 

2> 

CJ  O 
J  S 
UJ  O 
O  tr 

s< 

55 


U'S 

O 

a 

< 

« 

i/i" 

q: 

O 


O 

t3 

orT 

-j 

o 

oc 

y— 

Z 

o 

o 


CM 


56 


\ 

i 

1 

< 

I 

i 

j 


I 


Ficure  IV -Z — Automotive, 


WILL  BE  DESIGNED  BY  COMPUTERS. 


*85  BEYOND 


THE  FIRST  CONTROLLED  THERMONUCLEAR  (FUSION) 
POWER  PLANT  WILL  BE  DEMONSTRATED. 


Fkuu  IV-S.— SrMau. 


1.  It  assumes  a  socioeconomic  environment  left  to 
the  panelists  in  Probe  I.  That  environment  is  the  one 
assumed  for  TRW’s  1975  Long-Range  Plan. 

2.  It  will  focus  on  areata  of  prime  interest  to  TOW, 
however  romantic  and  jr'sational  other  possible  spec¬ 
ulations  may  seem. 

3.  Specialists  will  be  concentrating  on  their  spe¬ 
cialties. 

4.  It  will  be  extended  into  formal  planni  ig  as  prac¬ 
tical. 

5.  It  win  involve  members  of  TRW’s  technical  staff, 
and,  I  hasten  to  add,  on  a  spare-time  basis.  Hiinking 
of  the  kind  that  is  required  here  is  not  done  at  the 
expense  of  current  assignments.  It  is  almost  entirely  the 
result  of  off-campus  thinking  by  people  who  are  already 
well  acquainted  with  their  fields. 

For  figure  IV-6,  matrix  for  Probe  II,  panelists  have 
been  chosen  to  make  predictions  in  each  of  the  15 
categories  drown  The  categories  shown  on  the  left  were 
chosen  as  a  minimum  number  into  which  we  could  fit 
the  predictions  of  the  panelists  involved  with  Probe  I. 
One  or  two  cat^ories  have  been  added,  and  a  few  have 
been  combined.  The  rest  of  the  chart  illustrates  the  fact 
that  a  matrix  was  used  to  identify  areas  of  interest  to 
each  of  TRW’s  divisions.  Where  such  interest  was  indi¬ 
cated  by  group  captains,  one  or  more  panelists  were 
chosen  from  that  division.  By  this  method,  we  are 
hoping  to  involve  each  of  TOW’S  divisions  and  to 
^read  the  word  about  Probe  II,  making  it  a  part  of 
the  planning  of  every  division  in  the  corporation. 

Ibis  time,  we  are  asking  more  penetrating  questions. 

As  we  now  plan  it.  Probe  11  will  be  conducted  in  two 
rounds  plus  a  followup  as  required. 

1.  In  round  one,  panel  members  have  been  asked 
to  list  their  forecast  events  on  the  form  shown  in  IV— 7. 

Each  event  is  to  be  weighed  in  accordance  with  the 
factors  shown.  Desirability  is  to  be  considered  from 
the  viewpoint  of  the  customer.  Accordingly,  this  rating 
shculd  reflect  an  estimate  of  the  potential  demand 
which  would  indicate  the  importance  of  the  event  from 
a  marketing  standpoint.  Feasibility,  on  the  other  hand, 
is  to  be  considered  from  the  viewpoint  of  the  producer. 
It  should  reflect  an  estimate  of  both  the  technical  feasi¬ 
bility  and  the  difiiculty  likely  to  be  encountered  in  pre¬ 
requisite  developments.  Timing  should  reflect  both  an 
estimate  of  the  date  by  which  the  probability  is  0.5 
that  the  event  will  have  occurred  and  the  degree  of 
uncertainty  associated  with  that  estimate;  i.e.,  for  that 
date  by  which  there  is  a  reasonable  chance  that  the 
event  may  have  occurred :  p=0.2,  and  for  that  date  by 
which  the  event  is  almost  certain  to  have  occurred: 
p=0.9. 


2.  In  round  two,  each  panel  member  wiO  be  pro¬ 
vided  with  a  list  representing  our  composite  of  all  pre¬ 
dictions  of  his  own  panel  plus  those  of  other  panels 
which  also  relate  to  his  category.  In  this  round,  he  will 
be  asked  to  evaluate  all  events  with  respect  to  the  same 
three  factors  as  in  round  one.  Self-consistency  in  the 
list,  particularly  with  respect  to  the  order  of  anticipated 
dates  of  occurrence,  will  be  a  primary  objective. 

3.  In  an  informal  third  round,  resolution  of  differ¬ 
ences  of  opinion  will  be  sought.  Subsequent  attempts  to 
resolve  such  differences,  if  necessary,  will  be  conducted 
on  an  individual  basis. 

Present  plans  call  for  the  production  of  these  items 
(all  of  which  will  be  labeled  “TOW  Private”) : 

1.  A  series  of  monographs — one  for  each  category 
— ^for  distribution  to  panelists  involved  and  to  TRW 
employees  concerned  with  a  specific  category.  They 
will  contain  a  list  of  all  events  developed  during  our 
probe  which  may  have  an  impact  upon  TRW’s  opera¬ 
tion  in  that  particular  category. 

2.  Probe  II — a  document  similar  to  its  predecessor 
(but  more  complete)  in  which  each  predicted  event  is 
listed  only  once  under  that  category  in  which  the  impact 
upon  TOW  is  likely  to  be  the  most  significant.  Probe 
II,  as  such,  will  be  used  principally  by  group  executives 
and  their  planners.  For  the  convenience  of  these  people, 
predicted  events  will  be  indicated  for  group  attention^ 
in  columns  marked  for  that  purpose. 

We  have  still  to  deal  with  the  problem  that  major 
events  to  appear  in  Probe  II  will  be  of  little  use  to 
planners  unless  subdivided  into  more  manageable  units. 
We  found  that  Probe  I  was  similar  to  a  road  map  on 
which  only  the  locations  and  names  of  large  cities 
appeared.  The  predicted  events  were  so  all-consuming 
that  TRW  was  unable  to  participate  in  them  as  a  sole 
developer  and  financier.  The  roadmaps  containing  only 
large  cities  needed  further  development.  My  industrious 
assistant,  Don  Pyke,  invented  logic  networks  which 
plotted  the  roads  and  intermediate  cities  with  the  help 
of  technical  people  acquainted  with  the  areas  in  ques¬ 
tion.  Having  the  plots  of  intermediate  cities  of  impor¬ 
tance,  or  developments  prerequisite  to  the  final  events, 
engineers  pointed  out  to  Don  that  there  were  many 
fallout  products  which  should  be  considered.  I  look 
upon  these  as  interesting  little  villages  surrounding  the 
intermediate  cities,  and  it  is  in  just  those  villages  that 
TRW  will  probably  find  its  greatest  rewards.  It  is  Aere 
thsi:  the  food  is  grown  which  may  find  its  way  to  the 
big  cities. 

By  this  time,  we  had  dubbed  the  charts  with  the 
highways,  intermediate  cities  and  villages,  “SOON 
charts,”  the  word  being  an  acronym  for  Sequence  of 
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I  Opportunities  and  Negatives  for  the  company.  We 
I  decided  to  plot  logic  networks  and  SOON  charts  for  a 

I  couple  of  important  examples.  Our  first  covered  the 

j  event,  predicted  to  occur  about  1972,  “Holographic 
techniques,  utilizing  two  or  three  color  lasers  for  re¬ 
cording,  and  ordinary  white  light  for  viewing,  will  be 
used  to  produce  3— D  color  movies.” 

Figure  IV-8  outlines  the  logic  network  for  this 
event.  We  have  tried  to  identify  the  sequence  of  devel¬ 
opments  which  would  have  to  precede  the  occurren<% 
of  our  events  Those  in  the  upper  part  of  the  chart  relate 
to  the  development  of  lasers  and  pulsing  technology. 
The  middle  band  traces  developments  required  in  pro¬ 
duction  techniques,  while  the  lower  part  of  the  chart  is 
concerned  with  projection  for  small-  and  large-screen 
viewing. 

The  next  stqi  as  shown  in  figure  IV-9  was  to  expand 
j  the  network  to  include  the  logical  spin-oSs  which  might 
j  be  expected  by  way  of  corollary  developments.  For 
example:  / 

1.  As  a  result  of  the  development  of  techniques  for 
I  achieving  the  desired  intensity  and  coherence  of  lasers 
I  required  to  produce  holographic  movies,  exploration 

under  water  with  green  lasets  and  pattern  recognition 
technology  will  be  enhanc^;  laser  machining  tech¬ 
niques  will  be  improved;  and  complex  synthesis  of 
microcircuits  may  become  more  practical. 

2.  The  development  of  appropriate  pulsing  tech¬ 
niques  ^ould  open  a  number  of  new  possibilities  in 
stroboscopic  applications;  telemetry;  and  communi¬ 
cations. 

3.  The  development  of  rmording  media  should  lead 
to  improvements  in:  information  storage  capabilities; 
filter  and  light  technology;  high-speed  photography; 

I  and  text  illustrations. 

4.  The  development  of  techniques  for  the  protection 
of  actors  and  viewers  should  contribute  to  defense 
against  laser  weapons. 

5.  One  can’t  think  of  movies  today  without  con¬ 
sidering  television,  and  a  logical  extension  of  the  pre¬ 
dicted  event  would  suggest  the  possibility  of  3-D  color 
holographic  TV  in  the  home.  Before  this  can  happen, 

I  however,  it  would  have  to  either  (a)  achieve  the  d>ility 

\  to  transmit  information  at  a  rate  of  approximately 

i  2.5x10**  bits  per  second,  or  (b)  develop  alternative 

j  techniques  which  would  eliminate  the  ne^  for  trans¬ 


mission  of  redundant  information.  On  this  SOON 
chart,  the  “N”,  or  negatives,  Jire  not  identified  as  such. 
They  are  the  threats  to  current  product  lines 

At  the  company’s  Vermont  conference  in  July  of 
this  year,  I  presented  SOOl^  charts  related  to  develop¬ 
ments  required  for  the  widespread  availability  of  dec- 
tric  automobiles.  The  specific  event  was  “New  personal 
and  mass  tranqiortation  vehicles  powered  by  battery- 
electric  systems  will  be  in  common  use  by  1980.”  The 
negatives  presented  at  that  time  were  associated  with 
.  the  fact  that  electric  automobiles  (excqrt  hybrids)  will 
use  no  valves.  We  are  the  leading  producer  of  auto¬ 
mobile  valves  in  the  United  States.  Thus,  the  electric 
automobile  poses  a  threat  in  that  area,  as  it  does  in  the 
possible  obsolescence  of  current  suspension  systems  and 
steering  mechanisms,  which  we  produce  in  volume. 

There  is  another  step  which  can  be  taken  in  SOON 
charting,  namely,  the  production  of  what  we  caD 
Phase  II  SOON  charts.  These  consist  of  the  superposi- 
tioii  of  those  portions  of  Phase  I  SOON  charts  related 
to  a  specific  technology.  This  should  help  identify  the 
key  developments  in  that  technology  which  are  com¬ 
mon  to  a  number  of  final  events,  even  though  TRW 
may  have  no  interest  in  those  events,  per  se.  Back  to 
my  analogy,  though,  the  villages  of  interest  are  those 
whose  produce  finds  its  way  to  several  cities. 

It  is  too  early  to  assess  merits  and  shortcomings  of 
Probe  II.  At  the  present  time,  however,  we  know  that 
enthusiasm  is  high  among  some  140  technical  people 
whom  we  have  instructed  not  to  neglect  their  day-to- 
day  job  commitments  in  the  course  of  imagining  the 
future.  We  don’t  think  that  this  will  occur.  In  fact,  we 
look  upon  technological  forecasting  as  a  very  healthy 
vision-extending  exercise  which  will  place  in  context 
the  current  activities  of  people  engaged  in  the  exercise. 

In  closing,  I  would  be  remiss  if  I  were  not  to  locate 
our  current  effort  in  the  framework  of  a  corporate 
planning  effort.  Figure  IV-IO  shows  the  locadon  of 
our  technological  forecasting  in  the  platming  cycle, 
with  a  little  art  work  thrown  in  to  riiow  that  it  is  a 
chicken-and-egg  proposition  as  claimed  in  the  Lake 
Placid  conference  presentation.  If  I  were  to  choose  one 
egg  whici;  had  to  precede  the  other  thiee,  I  would  pidc 
the  one  under  the  hen,  entitled,  “A  description  of  prob¬ 
able  environment  of  the  future.”  From  there  we  can 
complete  the  cycle  clockwise. 
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Dr.  Ldistone.  I  un  a  little  concerned  about  your 
■tatetnent  essentially  that  the  market  analysis  is  done  by 
engineers,  litis  puts  the  engineers  in  a  superman  cate¬ 
gory  on  the  entire  job  and  particularly,  of  cour^i  there 
are  the  niinengineering  aspects  of  the  oivironment,  the 
market  and  so  on.  Engineers  hare  been  wrong  in  the 
past  and  I  am  a  little  hesitant  to  rely  on  them  that 
much.  The  other  point  is  that  the  entire  approach  is 
tedioogy  oriented.  This  is  a  technological  forecast 
meeting  so  maybe  you  corered  this  part,  the  facilities 
for  planning  and  pmsonnd  derdopnMnt  u  a  mix.  By 
19B0  you  may  need  difierort  mix  of  people,  so  you 


don’t  have  to  worry  about  this,  corporate  goals  and  the 
facilities  among  other  things.  I  am  wondering;  Is  this 
integrated  and  the  other  aspects  being  done  in  a  com¬ 
patible  way  so  we  can  put  this  all  together? 

Dr.  Nobth.  1  think  we  obviously  have  not  done  the 
whole  job.  First,  let  me  talk  about  the  use  of  ei^neers. 
I  think  eveh  marketing  people  are  being  chosen  from 
the  ranks  of  technical  people  these  days  because  of  the 
large  technological  content.  Iheae  ieiiows  can  do  a 
pretty  good  job.  Graiited  they  have  been  wrong  in  the 
past  and  they  can  be  again. 


Dr.  Linstone.  I  am  len  concerned  a^out  the  market¬ 
ing  people  than  the  economists  and  the  sociologists. 

Dr.  North.  I  think  it  is  very  important  that  we  have 
it  in  there.  I  doubt  whether  the  economists  and  the 
sociologists  should  make  predictions  with  respect  to 
the  technical  events.  They  certainly  should  fe^  into 
the  background  with  respect  to  the  socioeconomic 
environment.  The  other  comments  are  certainly  valid, 
that  is,  that  more  needs  to  go  into  decisions  to  proceed 
than  just  the  technical  factors.  Our  company  is  a  little 
different.  We  are  highly  decentralized  so  that  each  of 
the  four  groups  has  its  own  problem  and  laws  of  re¬ 
sponsibility  and  the  screws  are  put  down  on  them  pretty 
hard  to  meet  their  goals.  Therefore,  before  they  would 
engage  in  anything — ^these  are  merely  suggestions  for 
them — ^but  we  are  counting  on  them  to  do  their  own 
charting  or  use  the  information  in  any  way  they  see 
fit.  If  they  are  not  using  it,  we  will  complain  and  say, 
“You  should  be  using  this  and  here  are  some  sugges¬ 
tions  to  guide  you.”  It  is  going  to  be  up  to  them  to 
introduce  such  things  as  markets,  availability  of  plant 
space,  availability  of  engineers,  plant  workers,  and 
whi**  have  you.  We  are  not  trying  to  put  those  factors 
into  it.  TTiey  will  be  parts  of  the  business  decisions 
made  in  the  selection. 

Mr.  FhiEDMAN.  Do  you  have  any  data,  historical  data 
primarily,  upon  which  you  can  base  the  validity  of 
your  tech  forecasting?  For  example,  something  yoL 
might  have  thought  of  S  years  ago  and  today  stop  and 
look  at  what  happened? 

Mr.  North.  No,  it  is  too  young.  This  is  only  a  2- 
year-old  project.  I  may  not  et'en  want  to  come  to  the 
meeting  next  year  because  they  do  so  poorly,  but  I 
don’t  think  it’s  going  to  do  that  badly.  We  have  seen 
within  this  2  years  some  things  change.  All  of  them 
have  been  brought  in  closer,  and  all  the  changes  we 
think  should  be  made  now.  I  think  people  have  found 
in  the  past,  who  haven’t  done  things  this  way,  that  they 
have  been  too  conservative  in  their  estimates.  I  have 
no  background  figures  on  it. 

Mr.  F>;iiEDHAN.  Do  you  have  any  i.iformation  con¬ 
cerning  that  which  affected  your  forecast  most,  things 
which  perhaps  suggested  what  you  have  forecast  is  not 
exactly  the  rule?  Was  it  economic  within  the 
corporation? 

Dr.  North.  You  mean  things  that  brought  it  in 
closer? 

Mr.  Friedman  Yes. 

Dr.  North.  No.  Hie  things  I  was  referring  to  were 
the  technological  pace  being  greater  than  we  thought. 
I  mentioned  in  passing  the  microelectronics  in  tele¬ 
vision  that  costs  have  dropped  much  faster;  prices  have 


drooled  much  faster  than  I  would  have  predicted.  1 
think  that  is  the  all-encompassing  factor. 

Mr.  Friedman.  Have  you  come  up  against  a  situation 
where  someone  in  the  decisionmaking  place  in  the 
organization  simply  decided  they  did  not  want  to  put 
money  in  a  certain  area  which  later  turned  out  to  be 
or  could  have  been  profitable? 

Dr.  North.  Yes,  very  definitely.  Not  in  connecthm 
with  a  particular  forecast,  because  it  is  just  2  years  old, 
but  I  have  known  many  people  in  the  corporation  who 
have  forecast  “that  is  going  to  be  a  very  promising 
field.”  Only  last  week,  one  of  them  ::>ld  me  we  wouM 
have  in  that  area  instead  of  about  $50  million  sales, 
$250  million  sales  and  I  think  I  believe  him. 

Major  Martino.  I  believe  the  experiment  with  the 
use  of  the  Delphi  procedure  you  are  now  conducting  is 
the  biggest  in  history.  I  know  of  one  which  was  run 
with  a  panel  of  a  hundred  people  but  yours  is  even 
more  complex  in  that  you  have  apparently  c  little  over 
a  hundred  people  divided  into  several  panels.  I  am 
pleased  to  see  that  you  are  able  to  manage  it  with  two 
profes^iionals  part  time  and  some  administrative  sup¬ 
port.  This  is  very  interesting  from  the  standpoint  of  a 
number  of  other  tests  that  I  am  aware  of  to  -iheck  the 
feasibility  and  the  useabOity  of  a  i)elphi  method. 
Dr.  Dalkey  may  have  some  other  information  on  the 
size  of  the  panels  in  the  past,  but  to  my  knowledge  this 
is  the  largest  that’s  ever  been  run. 

Dr.  Dalkey.  This  is,  I  believe,  the  largest  study  both 
with  respect  to  the  number  of  participants  and  the 
complexity  of  the  kinds  of  questions  and  action  that  are 
involved.  I  must  say  the  most  interesting  statement  you 
made  is  when  you  went  on  from  Probe  1  to  Probe  2. 
From  the  standpoint  of  this  group,  I  think  this  is  very 
significant:  that  you  didn’t  find  it  expedient  to  quietly 
forget  it  after  Probe  1. 

Dr.  North.  Some  people  criticised  us  by  saying, 
“you  haven’t  rsed  Probe  1  yei.  Why  are  you  going  to 
Probe  2?”  My  re^nse  was  "I  ibdn’t  feel  that  the 
foundation  was  sound  enough  because  of  the  speed 
with  which  we  put  through  the  first  one,  and  it  was 
lacking  in  the  most  important  areas  where  we  needed 
it.”  I  might  add  that  I  don’t  think  we  are  going  to  have 
140  people  in  the  end.  There  are  many  people  who  just 
can’t  meet  deadlines  then  we  say  “forget  it,”  and  if  we 
end  with  a  hundred,  1*11  be  happy,  a  hundred  good 
thoughtful  inputs. 

Dr.  Slafkoshy.  I  couldn’t  help  but  agree  heartily 
with  Linstone,  but  in  terms  of  economists  and  sociolo¬ 
gists,  I  would  like  to  pose  the  question  very  simply. 
Why  did  you  only  use  engineers?  I  aaeame  you  have 
other  types  available,  systems  analysts,  scientista, 
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phyricinU,  nuthenutticians,  who  were  not  engineers, 
who  might  snake  better  mixes  between  what  is  now 
available  in  terms  of  bai^  and  applied  reeearch  and 
would  be  available  in  the  future.  Engineers  are  no* 
toricus  for  doing  things  the  way  they  were  taught  in 
school  and  doing  the  simolest  things  as  a  matter  of 
fact.  I  am  not  as  convinced  that  they  are  the  ones  who 
have  the  best  knowledge  of  basi'i  research  and  applied 
research  in  ascertaining  and  foreseeing  the  forefront. 
That  is  my  question.  ' 

Dr.  North.  First,  I  inclkded  scientists  among  engi¬ 
neers;  scientists  and  engineers  are  together.  Secondly, 
analysts  in  our  company  are  engineers  and  tliere  are 
a  lot  of  system  analysts  included.  We  don’t  have  many 
economists,  per  se,  on  our  staff.  There  are  a  few  and 
they  contributed  through  the  socioeconomic  back¬ 
ground  that  is  a  preamble  to  our  1975  plan  which  we 
are  also  using  for  this  guide.  I  don’t  think  we’ve  put 
to  bed  all  of  the  problems,  or  all  the  shortcomings. 

Dr.  Dalkey.  I  would  like  to  ask  a  couple  of  ques¬ 
tions  about  the  timing  of  the  exercise.  In  Probe  1  you 
wound  up  with  cat^orisation  of  events  and  with  the 
logic  network  tying  together  those  you  did  after  the 
Delphi  part  of  the  response.  As  I  understand  it  was 
done  by  a  small  group  of  analysts  working  with  that 
data. 

Dr.  North.  The  group  is  even  smaUer  than  that.  The 
'  inputs  we  received  from  panelists  were  rather  loose  the 
first  time;  some  were  unbelievably  loose,  such  as,  there 
would  be  quite  a  few  electric  automobiles  by  this  date. 
Don  Pyke  and  I  took  it  upon  ourselves  to  change  that 
to  read  whatever  is  read  here.  It  would  be  a  major 
factor  to  the  industry  and  will  go  as  far  as  to  say  that 
53  percent  will  be  electric,  but  we  sharpened  it  a  good 
deal  and  pai-ielists  in  the  second  round  did  a  inrther 
sharpening  job.  The  next  step  was  cross-index  fashion 
BO  a  man  could  look  at  the  portion  he  was  best  equipped 
to  handle —  there  were  some  40  categories;  not  15  as 
here.  Then  when  we  had  ail  the  results  we  said,  “Isn’t 
there  a  mall  number  of  categories  in  which  we  can  put 
all  of  these;  we’re  not  going  to  change  them,  we’re 
going  to  categorise  them.’’  Don  and  I  sat  down  and 
worked  up  these  first  15.  We  seemed  to,  by  a  little 
shoe-horning,  get  the  comments  in  one  of  them  and  we 
talked  to  the  people  subsequent  to  this  and  they  didn’t 
see  any  way — before  this  (probe  2),  we  ssid,  “Are 
there  any  categories  that  we  need  in  here?”  Yes,  there 
were  a  couple  to  be  added.  As  far  as  the  logic  network 
is  concerned,  tliat  didn’t  come  until  Probe  1  was  com¬ 
pleted;  the  early  part  of  thfr  year.  Probe  1  was  com¬ 
pleted  last  year. 


Dr.  Dalxey.  Secondly,  do  you  intend  to  indude  in 
Probe  2  any  coruideration  on  the  part  of  the  re¬ 
spondents  on  a  logic  network?  Are  you  going  to  let 
that  remain  until  after  the  resporrses? 

Dr.  North.  I  think  we  will  let  it  remain  and  not 
cause  the  panelists  to  call  out  their  pet  project  but  let 
the  management  exert  tdme  prerogative  as  to  what  they 
may  or  may  not  be  interested  in. 

Mr.  Cetron.  You  have  a  forecast  now,  of  what 
would  be  feasible  in  technology.  Now  how  do  you 
determine  what  you’re  going  to  do,  out  of  all  these 
possibilities? 

Dr.  North.  I  would  say;  that  out  of  the  total  picture 
which  will  be  indexed  for  groups,  group  planners  will 
look  it  over  and  decide  what  they  would  like  to  detail 
first.  We  will  give  them  help  in  detailing.  I  diould  have 
mentioned  earlier  I  don’t  think  these  SOON  charts  are 
the  only  way  of  going  about  this,  but  people  will  have 
different  ways  of  looking  into  this.  We  will  give  them 
help  where  they  want  it.  If  there  are  things  they  don’t 
look  at  that  they  should  have,  we  will  do  them  our¬ 
selves.  The  planning  of  the  functions  of  a  group, 
company,  or  division  is  a  function  of  opportunity  as 
well.  For  instance,  we  hope  that  some  thought  would 
have  been  given  before  a  person  makes  an  acquisition, 
let’s  say,  of  a  company  that  gets  them  into  a  whole  new 
field.  Is  it  really  promising?  Does  it  look  promising? 
This  should  be  used  there,  but  I  think  it  will  be  up  to 
those  people  to  use  it  as  they  see  fit  and  we  hope  it  will 
give  them  some  guidance.  That’s  just  where  it  stands 
now:  guidance  to  be  used,  if  useful. 

Mr.  Irwin.  A  little  bit  along  that  line.  Tlie  chicken 
and  egg  was  very  interesting  to  me  because  a  couple  of 
times  when  I’ve  gotten  involved  in  getting  the  demo¬ 
graphic  population  progression  with  an  economic  one. 
We  were  always  severely  pressed  by  the  men  who 
wanted  to  project  economics  for  a  population  pro¬ 
gression,  so  at  the  start  we  might  base  our  own  popula¬ 
tion  on  his.  I’m  not  completely  familiar  with  the 
procedures  of  the  National  Planning  Associati  but 
they  seem  to  solve  this  by  going  around  the  circle  once 
or  twice.  My  question  would  be:  Is  that  a  good  way 
to  handle  it?  It  doesn’t  matter  so  much  which  part  of 
the  circle  you  started  but  maybe  you  ought  to  make  a 
circuit  or  two. 

Dr.  North.  I  think  that’s  true.  Any  corporation  that 
does  this  has  pretty  good  ideas  of  what  kinds  of  busi¬ 
ness  they  are  going  to  be  in,  so  the  top  one  is  filled  and 
then  you  go  around  the  circle  or  back  up  and  realise 
you  don’t  have  enough  foundation  and  then  move  back 
to  that,  and  a  lot  of  that  amounts  to  going  around  the 
circle. 


Dr.  Lipstone  has  pioneered  in  devising  an  approach  to  forecasting,  known  as 
Mirage,  and  has  applied  this  technique  iteratively  over  a  series  of  years.  With  three 
completed  Mirage  forecasts  behind  him,  he  is  in  an  excellent  position  to  discuss  the 
situation  of  a  forecaster  who  has  to  live  in  the  time  period  he  once  made  forecasts  for. 
His  description  of  what  he  learned  through  these  iterations  of  the  Mirage  procedure 
should  be  of  value  to  all  forecasters. 


— ^Editor 


THE  MIRAGE  STUDIES  AND  SOME  IMPLICATIONS 


Harold  A.  Ungtono,  Lockhood  AIrtraH  Corp. 


The  requoet  to  diacuM  the  Mirage  studies  fills  me 
with  considerable  trepidation.  Studies  of  this  type  are 
part  science  and  part  art.  I  fear  that  analysis  of  such 
studies  is  likely  to  be  as  superfluous  for  those  who 
have  the  knack  as  it  is  inadequate  for  those  who  don’t 
have  it. 

Byron  Johnson  of  the  University  of  Colorado  tells 
the  story  of  the  philosopher  troubled  by  a  certain  prob¬ 
lem.  In  the  years  before  psychedelic  drugs  become 
common  those  desiring  to  expand  the  consciousness  of 
the  mind  were  forced  to  rely  on  primitive  substitutes — 
stiD  in  fairly  wide  use — containing  slcohol.  One 
philosopher  imbibed  freely  of  this  primitive  mind¬ 
expanding  drug.  As  he  sat  in  solemn,  but  u.isober, 
reflection,  he  realised  he  had  finally  found  the  key  to 
the  problem  which  bothered  him.  Fearful  that  he  might 
forgot,  when  sober,  what  had  been  so  clear  to  him  when 
drunk,  he  wrcrte  feverishly  upon  his  tablet  before 
turning  to  bed  to  sleep  it  off. 

The  next  morning,  as  he  pulled  himself  out  of  the 
fog,  he  realised  so'  ething  quite  important  had  hap¬ 
pened  the  night  before,  but  he  could  not  remember 
what  it  was.  So,  he  struggled  over  from  his  bed  to  his 
desk  and  there  he  found,  written  in  his  own  somewhat 
shaky  scrawl,  this  pregnant  sentence;  “Think  in  other 
categories.” 

This  is  precisely  the  thrust  of  our  effort  in  the 
Mirage  series. 

In  a  corporation,  each  division  invariably  undertakes 
planning  studies  of  its  own.  There  are  always  tactirn’ 
plans  which  tend  to  be  dominated  by:  (a)  medium- 
and  short-range  projections,  and  (b)  extrapolation  of 
current  lines  of  business  or  normal  system  improve¬ 
ment  cycles. 


Strategic  planning  may  occur  at  both  the  division 
and  corporate  levels,  or  at  the  corporate  level  only, 
depending  on  the  degree  of  decentralization  of  the 
organization  and  the  size  of  the  particular  division. 
This  planning  requires  a  long-range  perspective  and 
must  be  concerned  with  means  to  achieve  tire  growth 
and  evolution  set  forth  in  the  statement  of  objectives 
of  the  corporation. 

Stimulation  of  imaginative  interaction  between  new 
needs  and  new  techno-o;jy  is  one  major  element  in  this 
process.  In  other  words,  our  thinking  has  “opened  up”. 
Sucli  endeavors  should  help  identify  “good  risks”  and 
“poor  risks”  in  a  future  environment  characterized  by 
great  uncertainty. 

The  timing  of  the  studies  has  been  as  follows; 


Mirage  70”> . 

Performed 
....  195^ 

Subject 

Perief 

1965-70 

Mirage  75- . 

.... _ _ 

1963-64 

1970-75 

Mirage  80 . 

...  1965-66 

1975-80 

Although  the  studies  exhibit  differences  in  execu¬ 
tion,  their  general  approach  and  content  is  similar.  The 
MIRAGE  75  study  also  include  a  supplementary  “mis¬ 
sion  flow  chart  approach  originally  developed  by  the 
author  at  the  Rand  Corp.  (b)  This  model  was  ex¬ 
panded  in  a  study  performed  at  the  request  of  the  U.S. 
Army  Electronics  Command  in  parallel  with  MIRAGE 
30.  (3) 

It  sh  idd  be  noted  that  Mirage  70  can  be  assessed 
todav  in  hindsight:  I;  concentrated  on  the  period  in 
which  we  are  living  today. 

In  commenting  on  these  studies  I  shall  view  them  in 
toto. 


I,  OfNiRAL  FIATURE? 

A.  !n-HouM  Sponsorship.  can  be  maintained  in  this  manner.  A  disadvantage  is 

All  studies  have  been  done  with  private  funding.  the  unavailability  of  much  pertinent  analysis  data. 

The  foremost  advantage  is  that  complete  objectivity  Mirage  80  was  a  of,e  year  effort  with  an  average  of 
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15  technical  people.'  Obviously,  a  small  organization 
has  neither  the  financial  nor  manpower  resources  to 
undertake  such  a  project  on  its  own. 

B.  Organization. 

The  focal  point  of  each  study  has  been  at  the 
corporate  level. 

In  the  case  of  Mirage  80,  the  study  was  authorized 
by  the  president  of  our  corporation  and  his  staff  at  the 
annual  long-range  planning  meeting  in  1%5  and 
reported  at  the  same  meeting  in  1966. 

Stafhing  represents  a  major  challenge  and  few 
general  guidelines  can  be  provided. 

Youth  is  an  advantage — but  brilliant  young  Rhodes 
scholars  are  dilhcult  to  corral.  In  MiRAGE  70,  the 
oldest  team  member — the  author — was  35  years  old. 
The  seven-man  group  had  worked  as  a  unit  at  the 
corporate  level  for  several  years  before  tackling  this 
project. 

A  disadvantage  was  the  noninvolvement  of  the 
divisions,  making  implementation  of  recommendations 
difficult.  For  the  larger  Mirage  80  project,  an  ad  hoc 
team  was  formed  and  individuals  from  several  divi¬ 
sions  participated.  A  small  nucleus  remained  with  the 
project  throughout  the  year  but  most  participants  were 
with  the  study  for  a  few  months  only.  The  group 
worked  physically  together  for  only  a  limited  time. 
However,  the  deputy  project  leader,  Dalimil  Kybal,  was 
located  on  the  premises  of  the  division  furnishing  the 
largest  le  /e!  of  support.  There  are  also  significant 
advantages — and  disadvantages — in  using  an  ad  hoc 
group. 

Advantages 

•  communication  with,  hence  implementation  by, 
division  is  facilitaterl. 

•  little  chance  of  stalcness  in  succeeding  versions 
of  study. 

•  wider  basis  of  Uvhnical  support  by  info.mallv 
tapping  other  expertise  in  divisions. 

‘TAm  is  by  no  mrans  the  larftfst  in-housf  effort  undrrtak-  ’t  by 
a  ’’nanufitrtnrer.  \tinner:poli5-Ifone^urir5  Projert  Pattern,  for 
example,  included  itill  timr  tec^niccA  personnel  .  .  .  plus 

Piflnv  outside  consultants'^  at  ti  cost  of  f**.4iiation 

IFeek:’  Pec.  28,  p.  -’A 

II.  THE  STUDY 

A.  The  Work  Plan 

Figure  V-2  shows  the  general  work  program  of  both 
M  ir-arc  ftO  and  the  p.arallei  mission  flow  chart 
app'-oarh. 

kev  feature  was  the  development  of  hvp«ithrsei!. 
Ttirse  were  potential  eonelu.sions  hammered  out  in 


Disadvantages 

•  difficulty  'll  developing  a  cohesive  team  (desir¬ 
able  individuals  often  unavailable) . 

•  scattered  physical  location  of  participants. 

C.  Contents 

The  chapter  headings  of  Mirage  80  are  r^resenta- 
tivc  of  the  content  of  all  studies. 

1.  Introduction 

II.  The  World  Environment  of  tb.e  Seventies 

III.  Basic  Needs 
iV.  Strategic  Lift 

V.  Tpctii  .d  Operations 

VI.  Strategic  Offense  and  IX^fense 

VII.  Conflict  Management  and  (>ntrol 

VIIl.  NASA 

IX.  Summary  and  Conclusions  ^ 

X.  Implications  for  Ixickheed  fgep''ately  bound 
volume  for  internal  circulation  only) 

In  addition,  a  total  of  21  backup  reports  was 
produced. 

It  is  evident  that  the  studies  are  restricted  to  national 
securitv  areas,  i.e.,  DOD  and  NASA. 

A  schematic  of  the  structure  of  the  Mirage  80  study 
shows  the  logical  relaticoships  (figure  V-rl). 

The  study  of  the  environment  I’ch.  ID  focuses  on  the 
widening:  gap  as  the  most  crucial  issue  for  the  1970-80 
period.  This  leads  to  an  emphasis  of  problems  at  the 
extremes  of  the  conflict  spectrum: 

(a)  unsophi.siicatcd  warfare  involving  the  under¬ 
developed  areas. 

fb)  .sublimated  technological  warfare  involving  the 
s-’perpowers.* 

The  search  for  new  concepts  then  leads  to  tlie  “de- 
velopm.  lit  force”  atid  “rapid  action  control  :  1  strike 

fc.  .’e”  fch.  Ill). 

The  following  four  chapters  fch.  IV— VH)  consider 
the  operational  needs  and  new  technological  possibili¬ 
ties  in  the  recognized  military  categories. 

Chapter  VIII  takes  up  the  NASA  programs  and  the 
summarv  and  conclusions  form  chapter  IX.  The  stress 
is  on  good  risks  and  on  the  crucial  technological  bottle- 
neck<  in  satisfying  the  needs  for  the  197(V-80  {leriod. 

‘4  super i>oH-rr  is  defined  as  one  han'nfC  the  capchilitv  of  an- 
nihilatini  other  country. 

PROCEDURE 

rarlv  roundtable  discussions  on  the  basis  of  the  needs 
and  tcriinologv  interactions.  Fhev  then  forniesi  the 
basis  for  analvsis  bv  small  teain.».  (dearly,  judgment 
had  to  be  exercised  at  every  juncture  to  maintain  good 
balance  in  the  cb„jce  and  allocation  of  res-vUtc-es. 
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ENVIRONMENT  RESPONSE  FORCES  OPERATIONAL  NEEDS 


FicintE  V-1. 


B.  The  Analysis. 

There  was  no  single  formal  model  or  methodology 
employed  for  the  various  hypotheses  with  exception  of 
the  supplementary  mission  flow  chart  analysis  de¬ 
scribed  below. 

The  reasons  are  twofold : 

(1)  no  generally  satisfactory  model  is  known  to  us. 

(2)  formalized  procedures  tend  to  constrain  free 
interaction  between  needs  and  technology. 

The  second  point  requires  amplification.  Formalized 
models  cannot  substitute  for  creativity.  The  morpho¬ 
logical  approach  to  new  concept  formulation,  proposed 
many  years  ago  by  F.  Zwicky  of  the  California  Institute 
of  Technology,  has  proven  (’"sappointing. 


War  gaming  is  a  fine  tool  for  teaching,  or  for  gather¬ 
ing  data  on,  known  tactics;  it  is  rarely  the  source  of 
new  ideas.  Fornmlization  tends  to  impose  a  certain 
rigidity.  Another  problem  with  tach  models  in  tech¬ 
nological  forecasting  and  planning  is  the  danger  of 
self-delus’on.  Dubious  input  combined  vrith  superfici¬ 
ally  complex  models  may  ultimately  be  more  misleading 
than  illuminating.  Our  point  is  not  that  all  such  ap¬ 
proaches  are  useless;  rather,  a  technological  forecasting 
and/or  planning  effort  should  not  be  based  on  any  one 
such  model  no  matter  how  sophisticated  it  appears. 

Thus,  we  observe  no  uniformity  in  the  tools  for 
analysis  of  the  various  hypotheses.  A  few  examples 
from  Mirage  80  will  suffice  to  indicate  the  diversity 
of  techniques. 
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Example  1:  Basie  Force  Needs — New  Concept 
Derivation. 

The  interaction  of  needs  and  technology  in  the 
analysis  of  scenarios  and  derivation  of  concepts  of 
operation  appropriate  to  the  future  environment  is 
facilitated  by  stipulating  a  variety  of  artificial  con¬ 
straints.  This  is  entirely  analogous  to  the  consideration 
of  alternatives  ^y  President  Kennedy’s  “kitchen” 
cabimA  in  the  Cuban  missile  crisis.  T.  C.  Soronsen 
reports  (4)  that  the  President  was  presented  not  merely 
with  the  conventional  ahematives  (1)  do  nothing,  (2) 
invasion,  and  (3)  air  strike,  but  with  an  equal  number 
of  unorthodox  courses  of  action,  (4)  diplomatic  pres¬ 
sure,  (5)  secret  approach  to  Castro,  and  (6)  indirect 
military  action  (blockade). 

The  imposition  of  constraints  is  an  effective  tech¬ 
nique  to  avoid  the  familiar  more  of  the  same  response 
from  the  user  and  to  elicit  more  imaginative  concepts 
from  the  technologist.* 

It  was  found  in  Mirage  80  that  two  capabilities, 
(a)  rapid  presence,  and  (6)  control  by  neutralization 
without  destruction,  are  central  to  many  scenario-re- 
qranse  combinations. 

A  list  of  neutralization  concept  candidates  is  shewn 
in  table  V-1. 

Example  2:  Imjdications  of  a  New  Mission. 

(a)  A.  rapid  reaction  control  capability  of  the  kind 
mentioned  in  example  1  may  prove  useless  if  con 

*/t  is,  for  example,  most  unlikely  that  the  V3.  response  to  a 
Vietnam-type  eonfiia  situation  in  the  197ffs  shoidd—or  trill— 
retemUe  the  current  response. 


tingency  planning  cannot  be  performed  with  compatible 
speed.  There  is  little  need  to  consider  a  SAC-type  strike 
force  response  speed  if  the  revision  of  a  contingency 
plan  requires  months.  It  is  therefore  necessary  to  study 
the  automation  of  planning.  Figure  V-3  illustrates  the 
result  of  one  feasibility  analysis.  There  is  throughout 
a  conscious  attempt  to  avoid  the  trap  of  considering  a 
new  concept  and  ignoring  the  needs  which  it,  in  turn, 
generates.* 

(6)  Land  barriers  are  indicated  in  table  I  as  one  of 
the  neutralization  concepts.  Two  avenues  are  pursued. 
First,  a  theoretical  Lanchester-type  analysis  is  per¬ 
formed  to  determine  the  relation  between  cost  of  troops 
in  combination  with  a  barrier  and  cost  of  troops  alone, 
to  maintain  equal  degrees  of  permeability  along  a 
certain  line  segment  (see  figs.  V-4  and  V-5).  Second, 
the  technology  is  explored  and  a  rich  lode  of  ideas  is 
struck,  suggesting  entirely  new  systems  possibilities. 

It  is  noted  that  there  was  at  the  time  (1965)  no  sign 
of  the  very  visible  interest  in  this  subject  which  is 
currently  displayed. 

Example  3:  Strategic  Offense/Defense — An  Option 
Matrix. 

The  problem  of  China  as  an  nth  country  leads  to  a 
most  difficult  analysis  problem.  As  Herman  Kahn  has 
pointed  out,  list  making  can  be  a  useful  tool  even  where 
quantitative  analysis  is  not  possible.  Figure  V-6  shows 
the  matrix  of  nth  country  and  U.S.  alternatives.  This 

*  The  search  for  interactions  between  new  capabilities  and  new 
needs  leads  to  other  areas  of  vital  concern,  e.g.,  "hiding  tech¬ 
nology." 


Table  V-1. — Some  Neutralisation  Concepts. 


Effect 


Moans 


L  Militaiv  ojwntional  capability: 

Inhibit  mobility _ Create  land  Sairiert  (lee  VA3) ,  force 

down  aircraft,  incapacitate  ar'  iwK 
and  rehiclet  (BW/CW),  atop  nhip 
meTemeBt  ie-g.,  antipropeOer  de¬ 
vices)  ,  jam  navigation  or  guidance 
tystema,  and  Mke  vehicular  en- 
ginea. 

Prevent  resnpidy  - By  above  meant,  plus  deep  raids  and 

guerrilla  warfare. 


Dkrupt  command  and 
control  capability. 

laalate  (e.g..  separate 
two  koatiie  forcca). 

PraveW  covert 
eparatiosss 

Wedkm  will  to  fight 
(iaclodiag 
pactSemiiM). 


Jam  comniunicstiont,  kidnap  leaders, 
and  deairoy  radio  equipment  (e.g., 
CW  against  electrical  wiring). 
Oeate  land  barrien  (tec  VAS),  jam 
oa«u..aaicatioaa,  and  qaaraatiae. 
Deatroy  aalu.ai  cover  by  fire  or  ether 
m^Bs  and  marking  o(  pereoancL 
Tempaiary  iacapacitalimi  by  noa- 
lethal  BW/CW  (a^..  LSD  and  pay- 
cholofical  warfart. 


Effect  Moans 

Capture  or  dismantle  Deep  raids, 
materiel  (e.g., 
nuclear  weapons) . 

2.  Population  sod  governments; 

Disrupt  governmental  Kidnap  or  asasainate  leaders  and 
control.  guerrilla  warfare. 

Weaken  will  to  fight.  See  item  1,  guerrilla  activation,  and 
communicalions  control. 

Weaken  economy . Blockade  and  embargo. 

J.  Natural  reoources; 

Animal  and  fish  Alter  ecology  by  local  climate  r'-idi- 

fication  or  introduction  o>  new 
species,  and  tiler  breeding  pattern. 
Agriculture  Divert  irrigation  water  temporarily. 

Energy  Capture  dams  and  iacaptcitale  power 

linen,  refineriesL 

Minerals  . Oiantpl  mining  will  sat  incurring 

peraaacal  danaagn. 


AUTOMATION  OF  MILITARY  OPERATIONS  PLANNING 


100'^; 


A.  CURRENT  PRACTICE  AT  MOST  COMMANDS 

70  30 


B.  CURRENT  PRACTICE  AT  SELECTED  COMMANDS 


20  20  60 


C.  CURRENT  TECHNOLOGICAL  CAPABILITY 


10  10  80 


D,  PREDICTED  1975 


MANUAL 


SEMI-AUTOMATIC 


COMPUTER  ASSISTED 


AUTOMATIC 


Fkuw  V-3 


nutrix  has  so  {>i  dafied  solution.  It  hss  not  b«en  pos¬ 
sible  to  determiiic  a  good  way  for  the  United  State*  to 
divide  any  girea  ^budget  among  the  available  altenia- 
tives  in  light  of  the  threat  uncertainties,  although  it  is 
evident  that  thoe  are  good  and  poor  mixes  of  U.S. 
options. 

On  the  other  hand,  the  United  States-Soviet  stritegic 


offense/ defense  trade-offs  to  1980  have  been  subjected 
to  dtiailed  quantitative  analysis. 

Fxamp’-  4-  S'  Megic  Lift. 

The  steps  in  the  analysis  process  in  this  case  are  r  i 
follows. 

Su  •  V. — Review  of  Mirage  70  and  75  analyses  ni 
DOD  program  in  this  area. 


76 


OUBRRII.L.A  ATTACK 


V)  I  l/> 


8  18 


77 


(BARRIER  PERMEABILITY)^ 

(Cb  la  the  coat-e££ectlveaess  ratio  of  troop  element  lethality  to  barrier  element  lethality. 
Barrier  permeability  deflnea  the  ratio  o£  mobility  through  the  barrier  to  mobility 
without  the  barrier.) 


THEORETICAL  BORDER  SEALING  OPERATION 


COSTS 


COST-TROOPS  &  BARRIER 


COST-TROOPS  ALONE 


TROOPS 


ITROOP  REQUIREMENT- TROOPS  ALONE 


]/10 


ItROOP  REQUIREMENT-TROOPS  &  BARRIER 


f^imE  V-5. 


We  obeerve  in  Mirage  70  (1960) :  (r) 

It  it  a  paradox  that  woakneta  in  a  protfic  area — inuupoit — 
may  dratticaily  curtail  tht  entire  U.S.  limited  war  capability  in 
1965-70. 

This  is  compared  with  post- 1961  changes  in  DOD 
planning  for  this  field.* 

Step  2. — Development  of  h^rpotheaes. 

The  impact  of  authorised  new  systems  in  the  pviat- 
1970  period  is  considered.  The  family  of  scenarios  and 
new  force  concepts  in  the  environment  and  basic  needs 
discussion  are  assessed  in  terras  of  lift  req:  irements. 

utdicatiem  it  provijtj  in  thr  paNuunt  f'antt  4m€4 
SarWcai  Ctmmiltti  KtporU:  Airlift! ftaltll  farm  h<u$'t  in 


Orifiiifil  fbeml  fttr  62  0.9 

Fimml  ibcW  year  62  JJ 

FiK*l  yam  61 ..  1.4 

fiscal  ram  64  IJ 


f-jcW  ynr  65 
Fite.'f  yam  66 
Fiacot  ytmr  67 
Fiscal  yarn  64 


Tka*  wa  fmd  ikal  evrant  $paadia4  it  akaal  T0%  aboaa  dm 
arifisml  fiacal  yam  1961  iadgat. 


For  example,  the  rapid  action  conlrol  ^d  strike  feroe 
must  perform  unique  tasks.  The  most  likdy  bottle¬ 
necks  in  deployment  for  the  post- 1970  period  are 
estimated.  These  migl'l  be  in  the  fono  of  software  or 
hardware,  tactics  or  '’lOP’s,  aircrail  or  shipa,  rapid 
dcpioy  .nent  cr  steady  state  lift.  The  result  is  the  form- 
ulr.tion  of  hypotheees  involving  a  novd  operational 
concept  in  /xpid  depoyment. 

Step  V — Analysis. 

Consider^^ie  quantitative  analysia  is  possible  in  this 
problem.  .\  typical  anslysis  concerns  the  conipariifoa 
of  thv-  -lew  rapid  dcployrcmt  modes  w'th  known  one 
it.  g..  mariner  ships,  G-5A.  pre-posi  ioning).  Sup^.vw 
a  lOO.OOO-ton  deployment  to  an  inland  area  tn  couidry 
Jf  is  to  be  scct'rapliabed  with  s  closure  juxe  of  3tl  df./s 
or  less.  Which  modt;  represents  the  lowest  cuat  aytteea? 
Recognition  is  given  tc  she  Isct  '.hat  «rta^n  aar  a»p- 
tions  have  s  critical  impact  ou  tSj  rtsek.  For  e&aaq^ 
the  precticabilHy  of  land  pro-poaitioaing  and  the  sfiow- 
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M A  COUNTRY  AMOU.S.  ALTERNATIVES 


(5) 


DETERRENT 


DEFENSE  OFFENSE 


U.  S.  COUNTER 


N*  COUNTRY 
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10 

S  2 
S  3 
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military  octlon 

Dotor  U.S,  uM  of 
nueloor  woopom 

Ottor  throotonod  country 
from  roquoftlnp  U  old 

Allloncoi  with  other 
eour^trlM  to  ihoro 
nueloor  eopobility 
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B 

hS 

■ 
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pi 

H 
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■ 

■■ 
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■ 

B 

EKolctlon  of  convofltionol 
conflict  by  N“  country 

Nueloor  fint  itriko 
ojolmt  'other'  country 

Rotollotion  for  U.S. 
nueloor  weopoo  me 


Oetperotlon  attack 
on  U.S. 

Catalytic  ettack 


Accidental  or 
unouthorlied  attack 


■ 

t 

■ 

1 

— 

P 

ISi 

iBi 

R 

A.H-. 

r»:.vi''NV 

• 

r^,V\.*, 

IHAOIMe  INOICATC*  CITIWATKO  ■lONiriCANCC  OF  COWBINATIONI 


FkoaaV-C 


lAii  pnaceHma  eanitnf  power  of  tbe  milxUry  tranqwrt 
aircraft  itave  a  very  aignificant  cfiect.  Next,  the  ctooare 
tiiM  ia  redooed  to  20  daye  and  10  daya  The  reeuha  may 
ba  deported  aa  in  figure  V-7.  Senakivity  of  the  reauka 
11  then  chasksd. 

Slip  4. — The  implicatioaa  of  the  reauka  are  coO' 
iidarad.  Oiaagea  in  other  parte  of  the  total  lift  ayetem 


may  be  neceaaary  to  make  the  preferred  akemative 
meaningfui  Examples :  Cargo  handling,  comprehenaive 
iogiatica  control  ayaten. 

Other  <Txamplea  of  took  which  have  been  need ; 

•  Hktorical  treiKk:  Ratio  of  initial  oj^Moing 
ground  foroea  and  ka  relatioD  to  hattk  oat- 
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STRATEGIC  LIFT 
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A. 


Fkhm  V-7. 


come;  relation  between  mobility,  firepower, 
and  dispersion  in  past  ground  wars. 

•  Simulation  of  tactical  operations:  Armor — 
antiarmor  combat  (assault  and  defense,  direct 
and  indirect  fire  support). 

•  Decision  diagrams:  Graphic  means  to  show 
indifference  points  in  decision  alternatives  with 
major  uncertainties. 

•  Program  analysis:  Analysis  of  programming 
under  uncertainty  with  determination  of  de¬ 
cision  timetable  which  permits  maximum 
flexibility  in  future  choices  (e.  g.,  NASA 
programs  plan). 

In  the  final  assessment,  two  factors  are  of  overriding 
importance:  judgment  and  quality  of  the  analysis.  A 
Rand-type  “murder  board”  provides  a  most  desirable 
review  technique  for  the  various  component  studies. 

C.  The  Mission  Flow  Chart  Analysis. 

The  supplementary  approach  grew  out  of  the  dif¬ 
ficulty  (1962)  in  coming  to  grips  with  needs  for 
general-purpose  forces.  In  this  field  every  system  can 
perform  many  tasks  and  every  mission  can  be  per¬ 
formed  in  many  ways.  Further,  in  any  given  mission 
the  glamorous  systems  tend  to  receive  major  attention 
while  the  most  crucial  bottlenecks  may  be  in  small,  un- 
glamorous  phases  of  the  mission. 

The  approach  is  summarized  in  figure  V-8.  On  the 
basis  of  the  environmental  analysis  a  hierarchy  of 
missions  is  defined.  The  scenario  suggests  alternative 
concepts  of  operation;  concepts  lead  to  strategic  level 
missions  and  these  to  operations  missions;  finally,  wc 
are  led  to  support  or  basic  service  missions. 

After  the  hierarchy  is  developed  an  anatomical  flow 
chart  is  created  for  each  mission.  The  chart  shov  s  (a) 
the  alternative  means  or  systems  for  pierforming  the 
mission  step-by-step,  and  (6)  the  location  of  major 
weaknesses  in  any  of  the  systems.  The  general  structure 
of  a  flow  chart  is  shown  in  figure  V-9  and  the  cor¬ 
responding  detailed  chart  is  reproduced  in  figure  V— 10. 
Columns  of  each  chart  indicate  the  sequential  activities 
while  the  rows  represent  alternative  approach^. 

Problem  areas  are  indicated  by  verti'-ally  written 
comments  superimposed  on  the  functional  flow  lines. 
Those  problem  areas  considered  most  critical  to  the 
mission  performance  are  further  emphasized  by  rn- 
closing  the  vertical  comments  by  arrows. 

It  is  found  that  the  most  incisive  results  are  obtained 
when  the  analyst  deduces  the  initial  version  of  the  chart 
structure  without  extensive  prior  external'  inr-  The 
analyst  can  intuit  rather  complex  chs.rts,  even  in  fields 


he  has  not  previously  been  close  to,  by  calling  on  his 
general  systems  experience.  After  development  of  the 
general  outlines  of  the  chart,  standard  source  material 
is  consulted.  The  chart-drawing  exercise  forces  the 
analyst  to  think  hard  about  the  mission  under  study, 
frees  him  from  standardized  approaches  in  a  given 
field,  and  acts  as  an  organizing  t:vol.*  Following  this 
step,  experts  are  consulted  (1)  to  pinpoint  the  crucial 
weaknesses  and  (2)  to  indicate  new  alternatives  made 
possible  by  technological  achievements.  At  this  stage, 
the  relative  weakness  ranking  within  a  mission  is  inde¬ 
pendent  of  the  need  of  the  corresponding  mission 
itself.'  In  general,  a  weakness  that  cuts  across  all 
alternatives  for  performing  the  mission  will  rank  high. 

Weaknesses  will  vary  in  nature  according  to  mission 
level.  Those  at  the  highest  level  pertain  primanly  tc 
policy;  those  at  low  levels  involve  both  hardware 
(e.  g.,  lack  of  equipment)  and  software  (e.  g.,  obsolete 
standard  operating  procedures). 

'lire  charts  greatly  facilitate  and  stimulate  the  dis¬ 
cussions  and  the  interaction  between  the  technologist 
and  user  or  needs  analyst.  They  act  as  a  provocateur 
for  brainstorming  purposes,  keep  the  discussion 
focused,  and  assure  that  weaknesses  are  not  confined 
to  the  glamor  items.  Not  infrequently,  after  several 
iterations,  the  final  version  of  the  chart  and  its  crucial 
weaknesses  yields  significant  surprises. 

If  a  relative  ranking  of  weaknesses  is  desired  the 
following  steps  are  followed : 

(o)  relate  the  missions  at  one  level  to  those  at  the 
adjacent  levels  by  means  of  relevance  numbers. 

(b)  assign  weights  to  all  scenarios  in  the  conflict 
spectrum. 

(c)  combine  the  values  in  (o)  and  (6)  'u  obtain 
mission  need  values  for  aii  missions. 

frf)  rank  the  c.-ucial  weaknesses  in  each  mission 
flow  chart. 

e)  combine  the  values  in  (c)  and  (d)  to  obtain 
relative  overall  weakness  values. 

Table  V_2  shows  a  sample  ranking  of  mission  (c) 
at  each  of  the  three  levels  and  'or  three  different 
scenario'  weightings  (b). 

'it  is  fill  s  nnt  unlikr  Hrtmer’s  rpnftxtual  map  (“Soria!  Tech- 
nol'ifrr"  Boi.i-  1966,  p.  ICj. 

ranking  nwr  deptna,  honttt'.  on  whu  ■.  portion  of  tkt 
S)’t€trum  tht  mission  ix  to  ht  used  with.  Tl.js  weakness  majr 
hr  rruriof  for  all  lo*t  inteniitY  ronflict  srenerios  but  no*  for  tiny 
hi^h  intensity  see  iarios.  A  division  of  ike  scenario  sprtfrum 
into  subsets  is  ihertfore  assumed,  and  the  normalised  wtel'  ,ss 
tvngkts  y  .  s,*€ci/ie  *  j^th  reference  to  the  applicable  scenario 
subztcls. 
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FLOW  CHART  ANALYSIS 


WEAKNESS  EVALUAnON 


PEACETIME  ASSISTANCE :  TECHNICAL 


Ficukk  V-9. 


^  o 

o 

^  tt: 

Q- 


.  -3 

OjJ* 

^  si 

CO  |s 

uj  is 


UJ  8c 

S=?  a  = 

<x«.  cs 

LiJ?|. 

0-53*“ 

“8 

i 


^  :i 


X  *  j  Q 

S?  LU 


x 

« 

% 

ill 

4 

SZi 

3il 

-i  -1 

iit  so 

lil  Si 

m 

=33 

§!£ 

<  if'U 

I'  -  if 

5?*  srf 

xii  III 

f*  1 

jlj 

iL 

\t% 

§  |5  3 

If  '  !l  ii  - 

_^1  1 

1  4 

iit  3l 

ocJ  S»s 

ii! 

CO 

LU 

S 

3 

•  3:: 
c-> 


RANKING  OF  MISSION  IMPORTANCE 


Mission  Cato0ory 

Siluatlons 

Case  I 

Case  II 

Case  III 

Strategic  Missions 

Emergency 

Neutralization 

Emergency 

preparedness 

(population  and 

preparedness 

Neutralization 

government) 

Strategic  warihre 

(population  and 

Counterinsurgency 

government) 

Peacetime  tcchni- 

cal  asi'  lstance 

• 

% 

Emergency 

e 

e 

preparedness 

• 

0 

• 

Operations  Missions 

Barriers 

Barriers 

Barriers 

Neutralization 

Neutralization 

Defense  of 

(military  opera- 

(military  opera- 

natural  resources 

tions  capability 

tions  capability 

and  natural 

and  natural 

resources) 

resources) 

Destruction  of 

Destruction  of 

Destruction  of 

ground  targets 

ground  targets 

ground  targets 

(by  air  and 

(by  air  and 

(by  air  and 

ground  action) 

ground  action) 

ground  action) 

• 

s 

• 

• 

• 

• 

e 

• 

e 

Service  Missions 

Combat  surveil- 

Combat  surveil- 

Communications 

lance  and 

lance  and 

Combat  surveil- 

reconnaissance 

reconnaissance 

lance  and 

Communications 

Communications 

reconn  >:i8>>anco 

Target  acquisition 

Target  acquisition 

Target  acquisition 

Intratheater 

Intratheater 

Electronic  warfare 

mobility 

mobility 

• 

• 

• 

• 

• 

• 

• 

• 

• 
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It  is  also  interesting  to  group  the  weaknesses  accord-  DOD  include  Joint  Chiefs  of  Staff,  Labs,  theater  com¬ 
ing  to  the  v^irious  levels  of  authority  that  should  be  mandcra,  etc.  Figures  V-1 1  and  V-12  aaggeat  the  con- 
oncerned  with  the  problem  and  its  solution.  Levels  in  tent  of  the  problem  end  solution  tnalrix,  (7) 


Authoi^t^ 


Activity 


FIELD  LEVEL 


GROUND  COMBAT 


Problem  Category:  (B5)  Lack  of  special  capability 

Problem:  Most  currently  available  barrier  devices  are 
destructive  in  nature;  sensor  barriers  are  not 
emphasized  enough 

Mission  Chart  Reference:  Chart  3. 10 

Impact  Level:  Important  in  low  and  medium  level  warfare 

Solution  Reference:  See  Matrix  below 


Fkdiie  V-11. — Problem  Matrix. 


GROUND  COMBAT 

DEPARTMENTAL 

LEVEL 

New  inlerservice  doctrine  will  be  required  if  extensive 
use  of  barriers  is  to  be  made.  For  example,  special 
coordination  will  be  rec|uircd  between  the  air  sower  and 
the  ground  troops 

FIELD 

LEVEL 

Special  field  tactics  must  be  devised  to  exploit  barriers, 
e.g. ,  the  relation  of  sensor  barriers  to  indirect  fire 
support  capability 

SYSTEMS 

PLANNERS 

Sensor  barriers  must  be  coordinated  into  overall  sur¬ 
veillance  and  perimeter  defense  systems 

LABORATORY 

LEVEL 

Research  on  sensor  barriers  should  be  emphasized 

FKumc  V-12.-^S«bliMi  Matrix. 


III.  SOME  COMMENTS 


A.  The  Forcing  Function — Need  or  Technology? 

It  was  noted  that  we  stressed  the  early  interaction 
of  needs  and  technology.  We  find  here  a  classical 
chicken-and-egg  argument.  Which  comes  first — the 
need  or  the  technology? 

Quinn,  in  the  “Harvard  Busin«w  Review”,  (8)  states; 
“Clearly  perceived  demand — not  excess  technological 
capacity — tends  to  be  the  primary  force  simulating 
technological  change.  In  fact,  a  technology  is  only 
utiliaed  if  it  responds  to  a  need.” 

Project  Hindsight  states:  (9) 


If  we  idenlify .  the  events  which  were  motivated  by  a 
clearly  perceived  DOD  need,  we  see  that  they  add  up  to 
95  percent  of  all  events. ...  It  became  clear  early  in  the 
study  that  technological  innovatiun  was  highly  oone- 
lated  with  need-recognith>a. 

The  problem  is  with  the  terms  “clearly  perceived 
demand”  and  “clearly  perceived  DOD  need."  They 
can  prove  quite  misleading.  Perception  of  a  new  need  is 
not  a  dramatic  peak  in  a  flat  landscape.  In  1939  the 
U.S.  War  Department  clearly  perceived  the  need  for 
tanks  but  not  for  atomic  bombs.  On  tlw  basis  of  re¬ 
search  advance  in  nuclear  physics,  it  was  the  lech- 
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nological  community  which  suggested  the  need  for 
atomic  bombs. 

Gear  perception  of  a  need  by  the  user  usually  occurs 
years  after  the  ideas  are  first  proposed  and  discussed. 
The  initial  inqietus  may  come  from  either  technology 
or  need  (fig.  V-13). 

Imagination  must  be  stressed  on  the  needs  as  well  as 
on  the  technology  side.  Several  years  ago  an  issue  of 
the  UJS.  Naval  Institute  Proceedings  *  was  devoted  to 
a  series  of  articles  on  the  Navy  of  the  IQTO’s.  Inter- 
speraed  in  this  issue  was  a  set  of  artists’  conceptions  of 
new  naval  systems  for  this  period  provided  by  the 
Martin  G>.  It  is  curious  but  significant  that  we  find  no 
relation  whatsoever  between  the  pictures  and  the  text. 
Whereas  the  former  are  hig^y  imaginative  technologi¬ 
cally,  the  discussion  of  needs  is  conservative  and  never 
refers  to  the  illustrated  concepts. 

In  1961,  I  was  invited  to  participate  in  the  Army’s 
annual  national  strategy  seminar  at  Carlisle  Barracks, 
Pa.  There  were  more  than  10  teams  of  officers  com¬ 
pleting  their  term  papers  on  a  national  strategy  for 
1971.  To  my  smaxement,  the  significant  differences 
between  these  efforts  were  in  such  items  as  numbers  of 
U.S.  divisions  in  Europe  (e.g.,  five  or  six  divisions  in 
1971). 

Part  of  the  difficulty  stems  from  the  education 
system.  Just  as  the  civilian  schools  are  slow  to  adjust 
to  changing  needs  (e.g.,  obsolete  engineering  curricula) 
so  military  schools  are  alow  to  alter  their  curricula  to 
place  proper  emphasis  on  material  suitable  to  the 
future  environment  in  which  the  graduates  will  operate. 
Today’s  academies  still  consider  wars  between  near 
equals,  characteristic  of  the  period  1800-1945  (e.g., 
Napoleonic  Wars,  World  Wars  I  and  II),  as  the  norm. 
For  the  period  of  the  seventies  and  eighties  this  seems 
strange  indeed.  We  observe  that  in  the  last  3  years  the 
world  has  witnessed  over  1,000  days  of  unconventional 
war  and  6  days  of  conventional  war. 

Another  source  is  the  traditional  approach  of  the 
State  Department  which  does  not  tend  to  encourage  the 
kind  of  research  on  fitture  worlds  done,  for  example, 
by  Herman  Kahn’s  Hudson  Institute. 

The  unsophisticated  warfare  iitvolvlng  the  under¬ 
developed  areas  is  not  likdy  to  restrict  itself  the 
guerrilla  warfare  now  familiar  in  South  Vietnam.  A 
sea  of  poor  surrounding  a  few  islands  of  prosperity 
win  develop  other  meana  There  are  already  some  clues. 
ExmmpU:  the  recent  Soviet-Chinese  border  incidents 
aloiig  the  Amur  River — thousands  of  unarmed  Chinese 
peasants  forced  at  (Chinese)  gunpoint  to  cross  into 
Soviet  territory.  (10) 

•Jmrnryim. 


Military  response  also  faces  the  complicating  factor 
that  the  poor  world-  has  a  2  : 1  population  advantage 
over  the  rich  world  end  by  1980  will  have  a  3  : 1  edge. 
On  the  other  hand,  we  vte  that  in  Vietnam  a  300,000 
enemy  force  (including  60,000  North  Vietnamese 
troops)  are  tying  down  1,200,000  U.S./RVN  troops.  A 
ratio  of  10  : 1  or  even  15  : 1  may  be  necessary  for 
military  victory — a  necessary,  but  not  sufficient,  con¬ 
dition  for  total  victory.  The  United  States  simply  does 
not  have  the  manpower  resources  to  invest  and  a  citizen 
a^y  may  be  unavailable  for  Vietnam -type  military 
duty  in  this  period.  Reliance  for  strictly  military  tasks 
may  have  to  be  placed  on  an  elite  professional  army. 

It  matters  little  whether  technology  or  need  provides 
the  original  spark.  But  it  matters  very  much  that  an 
imaginative  and  continuing  dialogue  between  the  user 
and  technologist  be  assured  so  that  we  do  not  find 
ourselves  with  future  technology  to  satisfy  a  current 
(but  not  future)  need. 

The  problem  of  planning  a  balanced  growth  for  an 
underdeveloped  country  system  without  the  stresses 
and  temporary  reverses  which  produce  mass  revolu¬ 
tions  is  today  largely  unsolved.  It  is  hoped  that  this 
challenge  can  be  met  by  social  technology  if  the 
massive  U.S.  intellectual  resources  are  brought  to  bear 
on  it.  The  point  is  that  the  bottleneck  is  on  the  tech¬ 
nology — not  the  need — side. 

(6)  The  very  rich  versus  the  rich. 

The  widening  gap  between  the  United  States  and 
the  Soviet  Union  as  well  as  Western  Europe  in  tue 
1970-80  period  suggests  two  alternatives  to  European 
or  Soviet  long-range  planners: 

(1)  Passive. — One  may  wait  for  the  historically 
inevitable  reversal  of  this  widening  gap.  The  pattern 
of  exponential  growth  followed  by  saturation,  so 
familiar  to  biologists  and  technological  forecasters, 
seems  to  apply  to  nations  as  well  as  to  cells  and 
technological  improvements. 

(2)  Active.— Mirage  80  considers  the  possibility 
of  a  close  tie  between  the  Soviet  Union  and  several 
major  West  European  countries.  This  could  lead  to 
“isolation  and  decline  of  the  United  States  by  severe 
economic  competition  in  Europe,  Asia,  Africa,  and 
Latin  America.  It  would  constitute  a  unique  way  to 
negate  the  widening  lead  of  the  United  States.”  (13) 
This  appron.  il  may  be  described  as  strategic  economic 
warfare  against  the  United  States. 

It  is  evident  that  here,  loo,  there  is  first  and  fore¬ 
most  a  requirement  to  marshal  our  resources  towards 
social  technology  to  address  this  potentially  serious 
threat. 
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TImm  two  examples  snfBce  to  illustrate  the  impending 
bottleneck  oa  the  technology  s.ide.  As  if  this  were  not 
enough,  we  must  add  that  the  entire  gap  problem  is 
grarely  exacerbated  by  the  inevitable  widening  gap  in 
communication  as  the  societies  grow  apart.  Eil*^ive 
I-allistic  missile  defense  against  a  Soviet  threat  was 
enunciated  as  s  clearly  perceived  need  almost  a  decade 
ago  hot  hard  science  technology  could  not  devdop  an 
adequate  solution.  We  now  face  an  even  more  serious 
challenge  to  find  answers  in  social  technology. 

B.  A  Crucial  Weakness:  Social  Technology. 

Our  own  analysis  of  needs  points  to  a  rapidly  grow* 
ing  dilemma.  The  new  needs  pose  requirements  on 
social  technology  *  which  we  may  be  incapable  of  meet¬ 
ing  due  to  lack  of  requisite  advances  in  the  social 
sciences. 

I  shall  illustrate  this  with  two  examples  from 
Mirage  80,  both  related  to  the  most  salient  char¬ 
acteristic  of  the  1970-80  period  the  widening  gap 
(see  fig.  V-14). 

(a)  The  rich  vs.  the  poor.  In  Mirage  80,  page  15, 
we  read: 

A  strictly  military  retponae  in  any  con6ict  inrolrina  the  tiader- 
dereloped  coontriea  it  at  beat  a  palliative.  Withoot  a  maaaive 
attack  on  the  an<ierlying  problema,  even  a  deeiaive  military  vic¬ 
tory  may  lead  to  an  equally  deeiaive  final  defeat  (e.g.,  Algeria). 

The  30  plua  advanced  counlriea  have  the  scientific  and  tech¬ 
nological  resources  potentially  adequate  to  alleviate  the  social, 
political,  and  economic  canaea  of  unrest  in  the  90  plus  nnder- 

*A  term  popalarited  by  OUf  Helmer  (ef.  “SocUil  Technology," 
Batic  Books,  1966). 


developed  countries.  However,  the  presmt  and  planned  efforts 
to  apply  these  resources  are  entirely  inadequate  to  forestall 
widespread  violent  upheaval.** 

We  are  not  simply  talking  here  about  larger  funding 
of  AID  and  Alliance  for  Progress  programs  or  about 
conscientious  native  reformers.  More  massive  assistance 
and  rapid  improvements  may  simply  increase  the 
viol^ce. 

There  are  striking  clues:  (a)  In  the  dozen  years  after 
the  death  of  Louis  XIV,  French  trade  increased  nearly 
100  million  livres — there  followed  the  French  Revolu¬ 
tion.  {b)  The  Russian  economy  experienced  rapid 
growth  in  the  period  1895-1913  (e.g.,  exports  5.5 
rubles  per  capita  1895-99,  9.1  in  1910-13 — there 
followed  the  Russian  Revolution.  (11)  (c)  Violence  in 
our  ghettos  has  come  hard  on  the  heels  of  the  mor*. 
dramatic  progress  in  civil  rights  legislation  in  50 
years. 

The  outstanding  research  by  I.  K.  and  R.  L.  Feiera- 
bend  (12)  and  W.  W.  Conroe  on  84  countries  for  the 
period  1948-62  clearly  supports  the  hypothesis  that — 

The  faster  the  rate  of  change  in  the  modemiiatiun  process 
within  any  giver  society,  the  higher  the  level  of  political  insta¬ 
bility  wtihin  that  society. 

“The  world  is  spending  over  $150  billion  per  year  on  military 
rslabiUhmenis ;  the  transfer  of  resources  from  developed  to 
nnderdevlepd  countries  is  $6-$8  billion  per  year.  There  is,  in 
addition,  the  time  element:  simple  calculation  shows  that  it 
takes  a  country  with  $100  GNP  per  capita  and  6  percent  growth 
rate  88  years  to  catch  up  with  a  $3,000  GNP  per  capita  country 
growing  at  only  2  percent. 
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GKP  (S  BILLIONS) 


Mr.  FhiEDMAN.  One  thing  that  prompted  us  was 
bnaed  on  needs,  not  technology.  Technology  was  de- 
vdoped  u  a  resuk  of  needs.  I  will  fortify  that  by  this 
statement.  The  Germans  were  developing  A.bombs  and 
if  we  didn’t,  we  would  he  in  trouble.  Therefore,  as  a 
resuit  of  the  need,  it  was  a  triggering  factor  and  had 
more  of  an  impact  towards  getting  action;  that’s  what 
I  am  talking  about.  It  is  true  that  later  it  becomes  hazy 
and  diflScuk  to  determine  where  the  beginning  point 

liea. 

Mr.  Cetron.  Is  there  a  near  orientation  or  a  tech¬ 
nological  feasibility?  Aren’t  they  made  up  of  both 
parts?  Even  the  A-bombs  are  part  of  both. 

Dr.  LnfSTONE.  Looking  through  forecasting,  a  clearly 
percieved  need,  not  mnate  technological  change.  This 
was  from  Professor  Quinn.  If  we  identify  the  events 
that  were  motivated  by  clearly  perceived  need,  they 
wruld  add  up  to  95  percent  of  all  events.  Early  in  the 
study  the  technological  innovation  was  correlated  with 
need  recognition.  What  do  you  mean  by  clearly  per¬ 
ceived  need?  This  is  a  critical  question  where  one  can 
get  into  a  tremendous  number  of  arguments. 

Dr.  Eber.  You  should  change  needs  to  deficiencies. 

Dr.  LmsTONE.  I  want  to  be  careful  about  future  de¬ 
ficiencies,  not  necessarily  current  deficiencies.  We  came 
up  with  10  years  as  a  solution  to  the  problem.  'Hiere 
will  be  other  problems  of  deficiencies.  In  Vietnam,  at  a 
manpower  ratio  of  4  to  1  over  the  enemy,  we  still  have 
problems.  Past  experience  indicates  it  ought  to  be 
10  to  1  for  success.  I  don’t  think  anything  has  happened 
in  Vietnam  to  disprove  this  ratio. 

Colonel  Luke,  I’m  trying  to  figure  out  that  chart.  I 
see  the  little  underdeveloped  countries  and  China  and 
it  is  on  record  that  we  are  going  to  spend  $5  billion 
because  of  something  done  by  a  country  which  has  a 
1150  per  capita  income.  We  have  Vietnam  and  don’t 
know  how  to  solve  the  problems.  It  seems  you  can  have 
this  but  in  the  context  of  combat,  to  the  people  of  the 
world,  it  doesn’t  mean  a  thing.  We  may  be  richer  but 
we  are  not  necessarily  smarter. 

Dr.  Linstone.  That’s  right.  The  fact  that  the  ghetto 
problem,  for  example,  surprised  so  many  people.  Why 
do  we  have  the  riots  right  after  the  biggest  improve¬ 
ment  in  civil  rights  legislation  in  50  years?  There  was 
rapid  improvement  in  many  past  prerevolutionary 
periods.  You  say  we  are  not  any  smarter  but  this  paper 
is  important  because  we  are  beginning  to  learn  some¬ 
thing.  First,  it  is  quite  apparent  we  don’t  just  spend 
money  in  these  areas  and  expect  to  close  this  gap.  Even 
if  we  could  get  Congress  to  appropriate  it.  it  would  be 
a  foolish  way  to  spend  the  money.  We  are  just  begin¬ 
ning  to  underhand  the  interactions  and  what  causes 


the  revolutions.  It’s  only  in  the  last  12  months  that 
things  have  been  published  on  the  problems  that  have 
been  involving  us  for  a  long  time.  The  importance  of 
looking  into  the  system  comes  in  gaining  understanding 
and  that  is  the  fir4  step  to  solution.  The  reason  for  the 
theory  in  one  paper  is:  When  you  get  an  improvement, 
hopes  go  up  and  people  assume  this  is  going  to  continue 
but  you  have  an  unbalanced  growth,  then  their  ex¬ 
pectations  are  upset  and  this  becomes  what  is  called 
the  revolutionary  gap. 

Dr.  Johnston.  This  appears  to  be  an  example  of 
technology  creating  need,  because  in  many  countries 
they  have  advanced  far  enough  so  that  every  peasant 
family  is  going  to  have  a  transistor  radio,  bicycle,  boat, 
and  even  TV.  The  point  is  they  have  found  this  gap; 
see  it;  are  aware  of  it,  even  though  they  can’t  under¬ 
stand  this  kind  of  chart.  Their  notion  of  wanting  a 
Mustang  10  years  after  they  got  bicycles  is  important 
to  understand,  and  they  think  more  about  transistors 
than  hunger. 

Dr.  Linestone.  When  you  begin  a  curve  and  essen¬ 
tially  have  to  take  off  at  a  point — that’s  when  your 
aspirations  are  high.  Further  research  may  contradict 
it  but  this  seems  to  be  central.  Coming  up  with  the 
crucial  needs  of  the  seventies  is  the  thing  that  needs 
number  1  priority. 

Mr.  Irwin.  I  was  interested  in  whether  you  have 
plugged  age-distribution  analysis  into  this  type  of 
thinking? 

Dr.  Linstone.  No,  we  have  not.  This  is  a  very  simple 
curve,  the  GNP  per  capita  type  of  thing.  We  have  to 
consider  for  China,  the  impact  of  changes  and  assump¬ 
tions  for  instance  population  control,  1  percent  per  year 
growth  instead  of  2  percent.  We  contradict  Professor 
Fuller  because  it  is  impossible  to  see  China  industrial¬ 
ized,  as  he  said,  in  1975.  Even  if  they  get  good  eco¬ 
nomic  growth  and  strong  population  control,  they  are 
still  under  the  $300  level. 

Mr.  Irwin.  The  point  is  the  last  20  years  have  seen 
the  most  dramatic  changes  in  age  distribution,  prob¬ 
ably  in  the  history  of  the  world,  by  enormous  levels. 
For  instance,  the  Japan  situation.  There  is  a  negative 
side  to  this  and  comment  was  the  Japanese  have  over¬ 
done  it  rapidly  by  lowering  the  birth  rate,  and  drained 
themselves  of  young  people  which  has  had  a  serious 
effect  on  the  whole  industrial  system  and  the  way 
people  think.  The  alternative  age  implications,  in  thoaq 
two  population  assumptions,  are  enormous.  You  are 
going  to  have  a  huge  mass  of  young  people  or  few. 
The  impact  of  military  and  government  employment, 
the  Air  Force,  has  created  a  tremendous  impact  on  the 
age  distribution  in  the  State  of  New  Mexico  and  there 
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are  so  many  children  in  the  5-  and  10-year-old  groups. 

I  imagine  I  will  find  they  are  having  problems  with 
schools  and  other  things.  I’m  thinking  in  terms  of  very 
general  psychology,  if  you  will,  mass  psychology.  The 
population — how  it  is  changed  by  these  alternative 
shifts  and  the  relative  number  of  people  in  different 
ages. 

Mr.  Cetron.  How  can  we  solve  these  problems  with 
inadequate  social  technology? 

Dr.  Linstone.  Wc  have  done  c  study  of  Sudan  due 
to  the  unpleasantness  last  year.  In  order  to  study  trans¬ 
portation  we  had  to  look  at  the  entire  national  planning 
and  the  entire  economy.  It  is  like  climate  control. 
People  used  to  make  tremendous  technological  fore¬ 
casts  in  the  1950’s:  How  to  retard  evaporation  and 
make  Florida  more  pleasant?  Then  it  occurred  to 
people:  What  will  happen  to  the  vegetation  if  this  is 
changed?  You  have  to  know  what  you  are  doing  here 
as  well  as  in  climatology  and  in  these  different  coun¬ 
tries.  You  don’t  understand  the  interactions  well 
enough  so  that  when  you  make  a  change  it  is  a  con¬ 
trolled  change.  An  article  I  read — I  think  was  in  the 
“Columbia  Business  Review,”  by  Pincus  from  Rand, 
pointed  out  that  the  most  effective  assistance  has  been 
given  where  the  countries  have  been  small.  Taiwan  is 
an  example.  I  suspect  one  reason  is  that  it  is  ea.sy  to 
understand  where  to  put  the  money  in  a  small  country, 
a  smaller  problem  than  India.  We’II  see  what  happens 
on  the  research.  The  social  structure  has  a  system: 
Where  do  you  put  the  money  and  the  technology?  This 
business,  as  Dr.  Johnston  pointed  out,  about  creating 
problems  with  technology  and  creating  revolution  by 
the  assistance  which  in  many  cases  we  tend  to  do. 

Mr.  Cetron.  Something  came  up  in  a  meeting  about 
4  months  ago  and  I  think  it  had  a  great  bearing.  Gen¬ 
eral  Marshall  said  the  war  in  Vietnam  is  going  to  be  a 
long  war  in  the  wrong  place  at  the  wrong  time — we  all 
agree  with  this.  A  briefing  was  given  the  Marine  Corps 
and  Colonel  Vaughn,  we  asked  him.  Where  would  you 
like  to  fight  least?  He  said,  “The  war  in  Detroit  is  the 
one  I  would  like  to  fight  least.”  Naturally,  anyone  with 
any  brains  is  at  least  smart  enough  to  know  you'll  fight 
the  war  you  are  not  prepared  for,  or  the  one  you  want 
to  fight  least  and  what  you  are  really  doing  in  all  your 
Mirage  is  using  an  old  adage  ol  “least  regrets."  Where 
do  I  lose  least?  If  you  are  not  building  a  capability. 

Dr.  Lin.stone.  Tire  point  is  that  you  have  to  do  the 
planning.  In  this  country  it  is  certainly  possible  to  have 
a  balanced  capability.  Isn’t  this  what  our  militan' 
friends  constantly  thought  about,  having  a  balanced 
force,  so  you  won’t  have  any  particular  weakness?  .Kn 
enemy  will  look  for  weak  spots  but,  what  must  be  done 


is  to  make  every  spot  equally  unattractive.  We  have 
superior  resources  to  look  into  these  problems.  In  long- 
range  military  documents  you  see  the  need  for  address¬ 
ing  these.  But  some  of  this  doesn’t  get  implemented;  it 
isn’t  attractive.  Unfortunately,  in  some  way  it  is  like 
the  problem  that  President  Eisenhower  talked  about: 
That  neither  military  nor  industry  find  it  particularly 
attractive  at  this  stage.  It  may  be,  as  Professor  Fuller 
alludes  to  all  kinds  of  possibilities.  Imagination  is  all 
we  are  trying  to  stimulate  and  in  a  sense,  in  a  small 
group  like  this,  we  shouldn’t  be — there  should  be  other 
groups  far  superior  in  addressing  the  military.  What 
always  strikes  me  is  when  I  gave  the  actual  Mirage 
briefing,  1  say  “these  are  new  missions  that  we  have 
considered.”  I  asked.  What  are  the  new  missions?  What 
are  the  new  kinds  of  things  you  want  to  do?  And  you 
get  a  blank  stare.  I’m  not  saying  these  are  correct 
missions  but  things  to  look  into  and  it  looks  like  there 
ought  to  be  significant  departures  from  the  current 
ones. 

Mr.  Cetron.  How  do  you  know  you  can  get  so  many 
things  in?  Red  China  sent  all  the  people  across  the 
border  and  if  they  kept  walking  until  they  reach  Viet¬ 
nam,  what  would  we  do  with  them?  You  can’t  put  that 
up  as  a  mission.  Yet  the  problems  involved  could  be 
horrible. 

Dr.  Linstone.  We  have  this  barrier  problem  because 
very  often  you  want  to  stop  large  masses  of  people  from 
moving.  You  need  a  different  approach  from  past  tradi¬ 
tional  approaches  to  the  whole  barrier  concept.  We  are 
making  some  beginnings  in  Vietnam.  It  is  a  different 
order  of  sophistication  in  the  thinking  of  the  subject 
and  the  China  problem.  You  just  can’t  handle  or  forgrt 
about  the  Chinese  masses. 

Dr.  Chacko.  You  feel  there  should  be  adequacies  in 
social  technologies.  I  was  trying  to  find  the  latest  word. 
You  referred  to  an  article  and  all  I  could  gather  waa 
there  is  a  gap  between  the  expectation  and  the  realisa¬ 
tion.  Is  this  the  extent  of  the  report? 

Dr.  Linstone.  The  other  phase  of  having  it  stay  at  a 
fairly  moderate  pace  doesn’t  have  the  crash  program 
funding  approach  it  deserves.  We  are  doing  a  lot  in 
Thailand  in  having  anthropologists  there  and  people 
who  are  soft  science  technologists,  researchers.  You 
can’t  solve  this  if  you  don’t  know  the  interactions.  L«* 
me  give  you  an  example  from  a  U.S.  Office  of  Education 
study  made  recently.  What  would  be  the  right  way  for 
legislature?  to  spei.d  more  money  in  depressed  areas? 
An  obvious  technological  view  is  build  attractive  school 
buildings  so  the  children  would  like  to  go  to  school. 
Somebody  else  would  point  out,  reduce  the  teacher- 
student  rstio  so  each  rtudent  gets  more  attention.  In 
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lUt  padibreaking  ctudv,  both  factors  were  minor  in 
a^ificanoe,  the  one  key  issue  was  the  attitude 
toward  education  in  the  home  and  everything  paled  to 
iaaigoiut  ence  in  comparison.  Ihis  in  a  sense  is  re- 
aeatch.  When  you  want  to  spend  money  every  problem 
at  the  preaeii:  time  is  quite  a  shock  to  a  number  of 
people  recommending  expenditures  for  new  school 
baiMinga.  It  came  as  quite  a  shock  to  the  legislature  in 
CaBfomia.  In  the  underdeveloped  areas  we  have  an 
obvious  gap.  We’ve  got  to  understand  the  problem 
better  than  we  do  at  present  in  order  to  come  up  with  a 
tedmological  approach.  There  are  the  quick-fix  solu¬ 
tions  “Yea,  we’ve  got  some  technology.  Let’s  give  every¬ 
body  a  TV  set,  modem  bathrooms,  and  so  on,  to  make 
them  happy.”  But  this  would  be  disastrous  because 
putting  money  in  the  wrong  way  g^  you  into  more 
trouble  than  you  were  to  start  with. 

Major  SwETT.  I  am  intrigued  with  what  you  are 
saying.  We  ran  a  study  similar  to  Delphi  except  we 
sorted  for  problems  instead  of  solutions.  The  problem 
is  identifying  future  Air  Force  missions.  It  boils  down 
to  a  demand  problem  which  is  the  same  phenomena 
you  are  calling  needs  versus  technology.  The  demand 
side  of  the  equation  rests  ultimately  on  the  theory  of 
utility,  human  values,  human  needs,  attitudes;  the 
areas  that  are  nominally  covered  by  the  social  sciences. 
The  supply  side:  Technology,  aircraft  capability,  et 
cetera,  rests  on  the  hard  sciences.  For  our  purpose,  the 
areas  moat  relevant  were  the  least  developed  and  the 
least  relevant  were  the  moat  developed.  On  social  tech¬ 
nology,  it  might  be  correct  to  say  that  technology  is 
baaed  on  a  science  and  the  present  status  of  the  social 
sciences  makes  it  a  mutually  contradictory  term.  The 
requirement  for  our  purpose  is  philosophy-of-science- 
type  bask  research  and  especially  the  concept  of  utility. 

Dr.  Slafkoskt  I  am  interested  in  one  aspect  of  the 
technique  used  with  the  Mirage.  You  implied  this  was 
primarily  your  in-house  people  who  were  part  and  par¬ 
cel  to  this.  What  association  did  you  have  with  the 
military?  Were  there  retired  military  working  with  you 
and  bow  was  the  military  side  brought  in?  The  reason 
I  am  asking  is  not  that  they  should  dictate  anything 
but  I  have  a  feeling  that  your  engineers  and  personnel 
may  not  be  as  familiar  with  military  operations  as  ther 
might  be  reqpiired  to  be  to  do  this  work. 

Dr.  LmsTONE.  Let  me  give  an  illustration  of  what 
happened  during  the  time  I  was  at  Rand.  We  asked  the 
Marine  G>rps,  when  we  were  working  on  the  flow  chart 
concept  and  gave  them  16  scenarios,  to  do  one  for  10, 
5,  IS  years  ahead,  not  a  6-month’t  scenario.  They  con- 
aoplad  to  come  up  with  the  rfsponses  for  each  scenario, 
and  wtuU  happmsed.  The  results  were  very  disappoint¬ 


ing.  The  returns  were  alwayp  the  same  familiar  ap¬ 
proach.  It  isn’t  just  the  Marine  Corps,  our  retired  mili¬ 
tary  are  the  same. 

Dr.  Slafkosky.  When^  I  was  involved  with  the  CNA 
(Center  for  Naval  Analyses)  on  a  study,  there  is  a 
broadening  just  in  the  Navy  Department,  in  terms  of 
having  officers  do  this.  CNA  has  available  approxi¬ 
mately  80  Navy  officers  who  are  parceled  to  each  study 
the  CNA  does.  They  are  actually  in-house  workmen. 
Marines  also  have  their  allotment.  This  study  was 
started  some  time  around  1960  or  1961  and  finished  in 
1963. 1  am  familiar  with  the  reluctance  to  do  th’  kind 
of  thing.  I  think  this  would  be  more  widely  recognized 
on  the  part  of  the  military,  certainly  those  of  us  con¬ 
cerned  with  studies  from  the  civilian  side  working  with 
the  military  to  open  these  doors,  considerably,  and  I 
am  sure  they  are  being  opened.  I  suggest  that  it  may 
be  a  new  way  of  getting  some  military  people.  You 
have  enough  military  people  to  help  you  and  they 
would  not  make  it  official  in  any  sense  of  the  word. 
The  idea  of  an  -'operational  concept  doesn’t  need  official 
sanction.  All  you  need  is  to  be  aware,  which  not  only 
makes  your  study  worthwhile,  but  more  attractive  to 
the  military  side. 

Dr.  Linstone.  There  was  an  issue  of  the  Naval  Insti¬ 
tute  Proceedings  in  1963  or  1964  which  was  very  symp¬ 
tomatic.  It  was  strange  indeed.  There  was  a  whole  issue 
devoted  to  the  Navy  in  the  seventies  including  articles 
by  admirals.  The  most  imaginative  one  was  a  Marine 
Corps  article  bv  Colonel  Saxton.  In  the  same  issue  were 
illustrations  of  new  concepts  provided  by  the  Martin 
Co. — and  the  interesting  thing  about  it  was  there  was 
absolutely  no  relation  whatsoever  between  the  new  and 
interesting  technological  ideas  and  the  entire  written 
material.  I  say  it  is  symptomatic,  because  this  is  the 
kind  of  things  we  were  up  against. 

Major  Martino.  Another  problem  mentioned  during 
your  talk  was  that  the  work  on  United  States-Soviet 
nuclear  interchange  had  been  studied  with  a  great  deal 
of  precision. 

Dr.  Linstone.  Compared  to  the  limited  war,  it  was 
an  earlier  analytical  problem. 

Major  Martino.  In  manv  problem  areas  which  have 
been  intensively  studied,  I  find  a  phenomenon  b^ 
de-<cribed  as  inbreeding,  where  a  stuilv  becomes  an 
art  form  like  the  modern  detective  novel.  There  is  a 
certain  way  to  write  a  detective  novel  and  if  you  depart 
from  the  standard  form  readers  won't  like  it  and  the 
same  thing  tends  to  happen  in  areas  that  are  studied 
intensively  by  the  same  people. 

Dr.  Linstone.  The  modeliiis  problem,  in  the  Mirage 
70,  we  studied  intensively  with  an  entirely  different 
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group  of  people  and  a  different  approach,  and  they 
stood  up  well.  We  are  happy  to  use  those  results.  They 
are  still  quite  reasonable.  You  are  correct.  The  in- 
breeding  is  to  be  avoided.  We  did  hot  use  the  same 
techniques  as  Rand,  but  the  emphasis  was  on  the  '«ction, 
reaction,  and  counteraction.  We  got  it  to  what  we 
called  technological  warfare.  We  went  right  to  1980 
to  our  scientists  and  said,  “We  have  a  multiple  inter¬ 
action  here;  as  a  scientist,  what  do  you  think  can  be 
done  on  birth  sides”  and  as  we  go  to  this  point,  the 
1980  point,  you  get  into  a  set  of  alternatives  for  de¬ 
fensive  action,  offensive  strategies  and  so  on,  and  when 
you  are  through  make  sensitivity  checks  and  realize 
certain  conclusions  you  can  draw.  I  admit  the  inbreed¬ 
ing  problem  is  always  there.  It  was  quite  a  problem  and 
I  don’t  know  if  we  were  entirely  successful  tearing  them 
loose  from  that  kind  of  orientation.  There  are  bound 
to  be  traces  of  parochialism. 

Dr.  Linstone.  We  learned  another  lesson  in  Mirage 
70  on  this.  For  example,  we  made  strong  statements 
about  guerilla  war  in  1959  and  1960.  Two  and  a  half 
years  later  people  asked  us  about  the  feasibility  of 
starting  a  little  guerilla  warfare  laboratory.  By  then, 
any  possible  advantages  you  might  have  gotten  were 
lost.  Everyone  is  a  prophet  except  in  your  own  house. 
It  always  looks  better  when  an  outsider  says  exactly 
the  same  thing.  The  only  reason  for  the  difference  be¬ 
tween  DOD  studies  and  our  study  was  we  were  not  paid 
by  DOD  money  and  were  really  interested  in  seeing  the 
outcome.  We  learned  a  lesson  from  the  first  study  and 
this  is  an  advantage  of  having  people  from  the  division 
take  part  in  the  study.  The  doers  are  more  closely  tied 
in  with  the  corporate  office  and  not  working  in  an  ivory 
tower.  Actions  have  been  taken  by  the  divisions  very 
much  in  li'  ■:  of  the  recommendations.  We  can  see  the 
evidence.  There  are  a  number  of  areas  where  things 
have  not  been  done  but  w  •  don’t  expect  a  perfect  bat¬ 
ting  average.  Obviously,  the  divisions  have  an  option. 
I  agree,  if  you  do  this  it  is  your  responsibility  to  be 
able  to  communicate,  and  if  you  can’t,  don’t  blame  the 
boss  for  not  understanding  and  not  taking  action.  We 
have  one  advantage — we  briefed  a  study  for  the  plan¬ 
ning  meeting  a  year  after  receiving  the  go-ahead,  to  the 
top  level  of  the  corporation  including  the  chairman  of 
the  board.  About  2  months  later,  at  a  regular  staff 
meeting  the  pr^ident  made  it  a  point  to  ask  what  we 
were  doing  about  this.  He  didn’t  say  “Are  we  imple¬ 
menting  everything?"  He  said,  "What  is  being  done? 
What  are  the  divitins  doing?"  We  had  to  have  a  com¬ 
mittee  of  vice  presidents  in  a  couple  of  the  areas  where 
we  were  making  recommendations  for  the  corporation. 
After  the  study  was  completed,  they  forttmd  a  council  of 

9i 


vice  presidents  to  look  into  what  was  being  done  and 
as  we  look  at  these  lO-years  plans  we  see  there  has  been 
an  impact.  It  has  been  particularly  strong  in  a  couple 
small  divisions,  because  they  don’t  have  the  opportunity 
to  do  this  themselves.  They  used  our  study  in  orienting 
their  resources  or  research  resources.  Definite  actions 
have  been  taken  in  the  large  divisions.  One  recommen¬ 
dation  had  nothing  to  do  with  anything  military.  The 
only  recommendation  was  to  completely  operate  in  the 
system  in-house  capability  in  your  own  organisation. 

Colonel  Vauchk.  I  was  on  a  Marine  Corps  long- 
range  study  panel.  It  was  formed  in  1963  u  an  ad  hoc 
group  by  our  commandant  who  gave  us  a  charter  to 
look  St  where  the  Marine  Corps  wants  to  be  in  1965; 
he  said  take  a  look  in  the  future  and  come  up  with 
some  ideas,  examine  everything,  and  it  possibly  mi|^ 
be  revolutionary.  Those  weae  his  exact  words  “poarihly 
revolutionary.”  He  made 'the  headlines  shortly  after¬ 
wards  for  his  idea  of  sending  a  battalion  of  marines  by 
rocket  anywhere  in  the  world  in  45  minutes  and  has 
concluded  that  by  1985,  it  was  technologically  feasible 
but  not  practical.  They  came  up  with  the  Marine  Corps 
of  1985  document  which  was  a  concept.  From  the  con¬ 
cept  they  derived  certain  objectives  which  was  the  start 
of  the  long-range  planning  process.  I  might  say  as  an 
aside,  in  looking  over  the  history  of  what  has  happened 
in  the  past  4Vj  years,  it  appears  there  has  been  a  tre¬ 
mendously  increased  interest  in  long-range  thought. 
A  few  years  ago  long-range  was  thought  to  be  5,  6  or 
7  years — later  up  to  10  and  has  jumped  to  20.  The 
Army  has  gone  up  25.  Dr.  Helmer  and  Mr.  Gordon 
from  Rand  go  even  further.  It  is  spreading  throughout 
the  entire  United  States,  but  one  thing  of  particular 
interest — informally  discussing  this  document  and 
came  up  with  the  concept  called  1965  which  was  pro¬ 
duced  about  1960.  How  often  should  we  do  this  and 
how  often  do  we  want  inputs?  One  of  the  prime  inputs 
was  the  Syracuse  University  Research  Corp.  The 
Marine  Corps  actually  got  the  contract.  I  believe  it  is 
used  as  a  basic  document  by  a  gr^t  many  people  called. 
“The  World  in  1985;  Technological  and  Environmental 
Forecast.”  Dr.  Linstone  you  mentioned  that  Lockheed 
has  a  lO-year  plan,  on  a  IQ-year  basis,  and  1  was  won¬ 
dering  how  often  this  is  revised  and  then,  if  it  is  not 
proprietary.  Since  you  have  gone  through  this  Mirage 
cycle  several  times,  do  you  have  any  feelings  on  how 
often  it  should  be  revised?  Do  you  have  any  comments 
on  whether  the  same  group  of  people  should  be  tatsd 
over  tnd  over  agsin  or  are  you  getting  this  inbreeding 
procma? 

Dr.  Lltcrro^cx.  1  think  the  biggeni  advantage  of  thla 
kind  of  center  corporate  offk* — this  kind  of  Mirage 
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•tady  and  having  people  from  the  division — is  at  least 
inbreeding  in  terms  of  a  long-range  planning  group,  is 
minimised,  Basically  it  is  a  help  if  you  use  a  different 
groiqi  the  next  time  except  possibly  one  or  two  for  their 
giq>erience.  There  tends  to  be  land  of  a  defensive  atti- 
to^  from  the  group  that  did  it  first;  it  isn’t  going  o 
coa|ilelely  contradict  itself  the  next  time.  It  is  like  a 
cmnadian  who  has  had  a  TV  show  for  a  long  time  where 
there  is  a  limited  nmnber  of  jokes  and  routines,  and 
you  have  exploited  them  in  one  study.  Yon  just  have  to 
get  another  rontiiie.  The  10-year  plan  is  a  fairly  recent 
innovation  which  is  required  as  an  annual  documrnt 
and  is  still  being  revised.  It's  an  evolutionary  process 
in  a  sense,  the  first  time  it  was  a  very  poor  document, 
the  next  time  it  was  better,  and  the  third  time  around 
batter  yet  I  say  poor  because  people  don’*  take  into 
account  the  uncertainties  and  problems.  We  are  begin¬ 
ning,  and  I  am  very  deeply  involved,  in  trying  to  im¬ 
prove  our  own  operation.  We  are  using  systems  analysis 
now  to  look  inward  rathe,  than  outward.  If  it  is  good 
enough  for  McNamara,  why  don’t  we  use  some  of  these 
tools  in  our  decisionmaking.  For  example,  there  has 
been  the  risk  analysia  which  I  might  say  the  aerospace 
industry  has  not  been  a  leader  in — it’s  been  outside  the 
aerospace:  risk  analysis;  decision  trees,  ell  these  kinds 
of  things,  all  these  kinds  of  tools  that  we  are  beginning 
to  use.  Some  amuieng  and  some  serious.  For  the  first 
tiiae  some  of  the  results  were  used  by  our  manageroenL 
We  weren’t  happy  with  this  10-year  plan,  for  example, 
they  came  up  with  certain  goals  and  essentially  filled 
in  the  businem  to  reach  these  goals.  We  asked  the 
question:  Isn’t  it  true  you  have  to  aim  for  a  much 


higher  levd  of  business  if  you  are  going  to  get  the 
level  you  are  talking  about?  You  know  you’re  not 
going  to  be  auccessbil  in  everything  you  do.  The  ques¬ 
tion  is:  What  u  your  range  of  certainty?  How  much 
do  you  have  to  aim  for  in  order  to  get  a  reasonable 
likdihood  of  getting  what  you  are  after?  This  shocked 
people  a  little.  Then  we  adced.  What  is  the  impact  of 
one  area  on  another?  For  example,  if  you  do  Project 
A,  does  this  increase  your  probability  of  doing  Project 
B  and  so  on?  I  think  Dr.  North  indicated  something 
similar  in  TRW  work.  They  were  it!  'ctant  to  do  this 
but  we  are  getting  better  each  time,  1  might  say. 
Regarding  the  substance  of  a  10-year  plan,  the  varia¬ 
tion  from  year  to  year  is  important  but  at  this  rtage  I 
don’t  think  it  is  the  key  element.  What  is  important  is 
an  exercise  and  each  is  a  trial  run  and  we  will  give  an 
improved  set  of  instructions  each  year. 

Colonel  Vauchn.  Your  mirages  were  spaced — ^Mirage 
70  came  in  1959  and  1960  and  then  1963,  196^-.  1965 
and  1966.  You’re  out  14  years  in  the  last  one.  Where 
do  you  think  is  your  peak?  How  far  out  do  you  want 
to  go? 

Dr.  LinsTONE.  Tin  not  particularly  concerned  about 
going  too  much  farther  out  at  this  point.  Within  a 
couple  of  years  we  might  want  to  look  up  to  1985  but 
at  the  moment  it  is  not  the  redoing  of  the  Mirage  study 
that  is  the  problem  but  finding  the  technology.  1  suspect 
if  we  did  it  over  again  now,  the  results  would  not  be 
much  different.  In  a  sense,  the  problem  is  m  ire:  you 
better  suggest  some  answers  than  state  the  problems 
again- 


Dr.  Johiuton’s  paper  looks  at  a  specialised  aspect  of  demography,  namely  the 
forecasting  of  the  labor  force.  The  Department  of  Labor  has  been  involved  in  this 
effort  for  many  years,  and  has  developed  an  approach  to  their  task  which  has  a 
number  of  aspects  that  might  be  of  value  to  forecasters  in  other  subject  areas. 

—  Editor 
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LONG  RANGE  PROJECTIONS  OF  LABOR  FORCE* 

Denis  F.  Johnston,  Bureau  of  Labor  Statistics,  U.S.  Department  of  Labor 


i.  SELECTED  TECHNIQUES  OF  PROJECTION 


One  of  the  more  intuitively  appealing  ways  to  discuss 
techniques  of  projection  is,  of  course,  to  range  them 
from  simple  to  complex.  Such  a  ranking  is  a  sensible 
pedagogical  device,  but  it  is  not  without  its  dangers. 
The  simplest  techniques,  when  applied  to  projections, 
are  commonly  labeled  “naive”  while  the  more  complex 
approaches  come  to  be  regarded,  at  least  by  implica¬ 
tion,  as  sophisticated.  While  the  users  of  such  terms 
might  deny  any  pejorative  intent,  the  connotation  that 
it  is  bad  to  be  naive  is  inescapable.  When  econometri¬ 
cians  refer  to  a  projection  as  naive,  they  mean  that  it 
has  been  developed  without  a  theoretical  grasp  of  the 
underlying  mechanism.  As  we  shall  see,  extrapolative 
techniques  are  typically  naive  in  this  sense.  However, 
the  term  is  still  unfortunate,  particularly  in  view  of  the 
fact  that  the  superiority  of  the  sophisticated  techniques 
is  by  no  means  clearly  established  either  in  economics 
or  demography,  at  least  according  to  such  conventional 
criteria  as  the  accuracy  of  the  results.  We  shall  there¬ 
fore  refer  to  simple  and  complex  techniques,  with  the 
admonition  that  true  sophistication  consists  in  knowing 
when  to  use  each  of  the  available  techniques. 

The  most  simple  projective  technique  is  to  assume  no 
change  over  the  period  of  the  projection,  from  the 
time  of  the  last  observation.  Such  a  forecast  (since  it 
provides  no  range  of  values,  it  is,  ’n  effect,  a  forecast) 
is  of  course  among  the  least  likely  lo  materialize,  since 
the  occurrence  of  some  change  is  surely  one  of  the  most 
dependable  of  expectations.  Where  change  is  so  gradual 
as  to  be  insignificant  over  short  periods,  a  forecast  of 
constancy  is  of  course  perfectly  defensible,  but  such 
circumstances  are  rare  indeed. 

Next  in  simplicity  is  a  projection  (forecast)  obtained 
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by  assuming  the  same  direction  and  amount  of  change 
over  the  projection  period  as  occurred  over  an  equiva¬ 
lent  period  in  the  past,  up  to  ti  e  date  or  base  line  from 
which  the  projection  extends,  iiy  this  simple  technique, 
if  the  labor  force  increased  by  12  millions  between  year 
* — 10  and  year  x,  it  will  be  projected  to  increase  by 
12  million  between  year  x  and  year  x-j-lO.  Although 
this  technique  at  least  recognizes  a  time  trend,  the  rate 
of  change  that  it  implicitly  assumes  is  of  a  very  par¬ 
ticular  and  implausible  nature — one  that  decreases 
regularly  through  time  with  the  increase  in  the  size  of 
the  base,  such  that  the  amount  of  increase  (or  change) 
in  each  period  remains  constant. 

Tlie  next  degree  in  complexity  is  introduced  by  as¬ 
suming  a  constant  rate  of  change  over  the  projection 
period.  This  technique,  like  the  one  preceding,  requires 
as  a  minimum  two  observations  in  the  past,  from  which 
an  average  rate  of  change  can  be  computed.  By  retain¬ 
ing  a  constant  rate  of  change,  this  technique  yields 
changes  of  gradually  changing  magnitude  as  the  size 
of  the  base  changes  through  time.  This  is  perhaps  the 
simplest  technique  enjoying  any  degree  of  verisimili¬ 
tude.  Among  human  populations,  rates  of  change  re¬ 
lating  to  many  social  and  economic  phenomena  do 
remain  approximately  constant  over  considerable  peri¬ 
ods  of  time,  particularly  if  we  ignore  minor  fluctua¬ 
tions.  As  we  have  noted,  the  above  techniques  are 
typically  employed  in  obtaining  forecasts.  However, 
such  forecasts  can  readily  be  modified  from  point  to 
interval  predictions  by  the  simple  expedient  of  intro¬ 
ducing  a  range  of  error  or  probable  variation  around 
the  specific  forecast  values.  Where  such  simple  tech¬ 
niques  are  called  for,  howm'er,  an  assumed  range  of 
probable  variation  could  only  be  ar.ived  at  by  guess¬ 
work. 

Somewhat  more  complicated  to  carry  out  are  the 
extrapolative  techniques  which  involve  superimposing 
a  curve  pf  some  kind  (of  which  a  straight  line  is  of 
course  a  limiting  case)  on  a  time  series  of  observations, 
deriving  the  forecast  by  reading  off  the  curve  where  it 
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intenecto  the  tiive  axis  at  each  forecast  date.’  Since 
moat  time  series  show  some  fluctuation  in  either  or 
both  the  magnitude  and  direction  of  changes  between 
BUocessive  ohserrations,  any  curve-fitting  technique  eu- 
taOs  some  kind  of  averaging  or  smoothing  of  these 
fiwtuations.  The  conventional  procedure  ia  to  fit  a  line 
of  least  squares  or  one  of  its  variants  through  tl^  given 
antes  of  observations,  reading  the  forecart  from  the 
extrapolated  line  in  the  usual  manner.  When  a  set  of 
projections  (or  range  of  forecasts)  is  desired,  it  is 
sometimes  possible  to  fit  such  lines  through  the  peaks 
and  the  troughs  of  the  given  observations,  thus  ob¬ 
taining  a  high  and  a  W  series  surrounding  the  basic 
extrapolation.  Unfortunately,  if  the  amplitude  of  the 
fluctuations  in  the  given  time  series  is  changing  through 
time  (i.e.,  if  the  data  are  heteroecedastic)  this  proce¬ 
dure  will  yield  absurd  results. 

i 

In  fitting  burves  to  any  time  series,  the  techrician 
must  seek  a  tradp'^off  between  a  curve  of  higher  order 
which  closely  follows  the  fluctuations  in  the  observed 
series  and  one  which  averages  out  these  fluctuations  in 
order  to  provide  a  more  straightforward  forecast.  ITie 
more  simple  curves  (including  the  straight  line  as  a 
limiting  case)  are  commonly  favored  intuitively.  They 
can  also  be  defended  on  theoretical  grounds,  insofar  as 
the  fluctuations  in  a  time  series  can  be  attributed  to 
irregular  changes,  sampling  variability,  the  occnrrence 
of  random  shocks,  and  the  like,  all  of  which  would 
tend,  in  the  longrun,  to  cancel  each  other  out.  Hence  a 
mioothing  procedure  which,  in  effect,  accomplishes  this 
cancellation  by  an  averaging  process  is  sensible.  Pro¬ 
jections  derived  from  higher-order  curves  which  at¬ 
tempt  to  capture  some  of  these  fluctuations  involve  the 
more  questionable  assumption  that  the  pattern  of  fluc¬ 
tuations  observed  over  a  finite  segment  of  past  time 
will  be  repeated  during  a  finite  future  time  period. 

Time  series  relating  to  social  or  economic  phe¬ 
nomena  commonly  reveal  a  strong  cyclical  pattern,  or 
else  the  operation  of  such  a  cycle  may  be  assumed  on 
theoretical  grounds.  In  such  cases,  the  technician  may 
wish  to  employ  a  higher-order  curve  which  reflects  the 
cycle  and  carries  it  forward  through  time.  This  usage 

‘There  it  considerable  literature  relating  to  the  fitting  of  expo¬ 
nential,  logistic,  and  similar  curves  to  time  series  of  data  on 
sSP^KSte  population  sixe  to  obtain  projections  of  total  popula¬ 
tion  for  different  future  items.  A  detailed  summary  of  earlier 
efforts  in  this  area  is  Joseph  J.  Spengler  (ISO)  For  a  critical 
review  by  one  of  the  rediscoverers  of  the  logistic  curve,  see 
Lowell  J.  Reed  (ISO).  More  recent  analyses  of  the  problems  of 
population  projection,  tempered  by  the  sobering  experiences  of 
the  WS^’s  and  1940’s,  include  Harold  F.  Dorn  (46) ,  John  Hajnal 
(81),  and  John  V.  Grauman  (6S).  For  a  balanced  summary  of 
current  developments  in  the  field,  see  Grauman  (66), 


represents  the  most  leguimate  employment  of  curve- 
fitting  techniques  to  obtain  extrapolations  for  projec¬ 
tion.  Unfortunately,  theories  supporting  the  existence 
of  cycles  in  the  field  of  manpower  analysis  are  typically 
too  general  in  their  formulation  to  provide  specific 
guidelines  as  to  the  type  and  phasing  of  the  cycles 
which  may  be  operative.^  Meanwhile,  extrapolative 
techniques  typically  suffer  from  an  absence  of  theoreti¬ 
cal  underpinning,  and  thus  represent  little  more  than 
the  assumption  that  the  future  will  be  an  extension  of 
past  trends. 

For  the  technician,  these  curve-fitting  extrapolative 
techniques  offer  the  sole  advantage  of  being  quick  and 
easy  to  employ,  and  readily  defensible,  at  least  in  a 
superficial  sense,  given  the  usual  “ceteris  paribus” 
clause.  However,  he  can  learn  very  little  either  from 
forecasts  which  prove  to  be  accurate  or  from  those 
which  fall  wide  of  the  mark,  since  his  efforts  are  not 
based  upon  a  theoretical  grasp  of  the  underlying 
mechanisms  in  operation.  For  the  policymaker,  on  the 
other  hand,  these  simpler  techniques  can  provide  a  very 
useful  check  on  the  reasonableness  or  plausibility  of 
projections  derived  from  more  complicated  procedures. 
The  basic  difflculty  with  any  extrapolative  technique  is 
its  inability  to  forecast  structural  changes  which  may 
modify  past  trends  quite  drastically.  The  central  prob¬ 
lem  in  long-range  projection  is  therefore  that  of  devel¬ 
oping  an  understanding  of  the  dynamics  underlying 
structural  changes  so  as  to  determine  the  likelihood 
that  such  changes  will  occur  over  the  forecast  period. 
A  theory  which  could  take  account  of  these  underlying 
factors  in  a  systemadc  fashion  would  provide  a  power¬ 
ful  tool  for  purposes  of  projection.* 

A  promising  extension  of  extrapolative  techniques  is 
the  technique  of  cohort  analysis.*  This  technique  can 
best  be  illustrated  in  terms  of  population  projections, 
where  it  has  already  proven  its  worth  as  a  substitute 
for  more  conventional  techniques  of  projection.  In  ex¬ 
trapolating  the  size  of  a  population  group  by  conven¬ 
tional  methods,  successive  observations  over  time  of 
the  size  of  each  particular  age-sex  group  are  obtained.  i 

’  Significant  progress  has  been  made  in  this  area.  See  Simon 
Kuznets  (102)  and,  more  recently,  Richard  A.  Easterlin  (48). 
Easterlin’s  more  extensive  work  on  this  subject,  entitled  “Popu¬ 
lation,  Labor  Force,  and  Long  Swings  in  Economic  Growth — 
The  American  Experience,"  is  forthcoming. 

*On  this  issue,  see  V.  I.ewis  Bassie  (7)  and  Rex  F.  Daly  (36). 

VA  useful  summary  of  the  application  of  cohort  analyds  in 
projecting  the  fertility  of  a  population  is  provided  by  Jacob  S. 
Siegel  et  al.  (I7I),  pp.  12-23.  The  technique  is  also  described 
in  Pascal  K.  Whelpton  (.200).  Pioneering  work  in  this  sria  in¬ 
cludes  the  study  by  Whelpton  (201)  and  by  Norman  B.  Ryder 
(757).  For  a  detailed  evaluation  of  the  cohort-fertility  method 
of  population  projection,  see  Whelptpn  et  al  (202). 
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From  this  time  series,  a  measure  of  its  average  rate  of 
change  over  the  base  period  is  obtained  and  applied  to 
the  latest  available  estimate  to  yield  a  projection  for 
the  desired  date  in  the  future.  Since  the  age-sex  cate¬ 
gory  to  which  the  successive  observations  relate  remains 
fixed  throughout  the  base  period,  the  actual  population 
observed  at  each  point  in  time  is  changing  constantly. 
In  contrast,  a  cohort  analysis  involves  successive  ob¬ 
servations  of  the  same  population  group  (or  cohort) 
as  it  passes  through  time.  For  example,  the  number  of 
persons  aged  10  to  14  in  1940  is  linked  with  the  num¬ 
ber  15  to  19  in  1945,  20  to  24  in  1950,  etc.®  The 
obvious  advantage  of  cohort  analysis  as  e  source  of 
trends  to  be  used  for  projection  is  that  each  cohort 
fcries  represents  the  actual  historical  experience  of  a 
specific  group  of  people  with  respect  to  the  parameter 
being  estimated.  This  provides  greater  theoretical  justi¬ 
fication  for  the  assumption  that  past  trends  will  con¬ 
tinue  into  the  future. 

The  principle  disadvantage  of  this  technique  is  that 
it  requires  a  great  deal  of  historical  data  of  comparable 
quality  with  observations  at  frequent  intervals.  Decen¬ 
nial  census  data  are  generally  unsatisfactory  for  this 
purpose,  since  they  yield  observations  only  at  10-year 
intervals.  However,  it  is  possible  that  useful  data  might 
be  obtained  from  a  series  of  annual-average  observa¬ 
tions  of  the  current  population  survey,  classified  by 
single  years  of  age,  provided  that  the  sampling  vari¬ 
ability  of  such  disaggregated  data  would  not  be  pro¬ 
hibitively  large.® 

The  experience  gained  by  demographers  in  cohort 
analysis  during  the  past  15  years  should  serve  as  a 
warning  against  its  uncritical  application  in  areas 
where  the  available  data  are  less  adequate  than  in  the 
field  of  population,  where  its  use  has  given  rise  to  a 
number  of  problems  despite  the  apparent  wealth  of 
available  data.  Nevertheless,  with  the  improvement  in 
both  the  quality  and  quantity  of  data  that  might  be 
expected  in  the  future,  techniques  of  cohort  analysis 
offer  great  potentiality  for  the  development  of  improved 
projections  in  a  number  of  areas  related  to  population, 
including  labor  force.' 

'A  “birth-cohort”  or  “age-cohort”  is  a  group  of  persons  bom 
within  a  specified  time  interval  {usually  1  or  5  years).  Simi¬ 
larly,  a  marriage-cohort,  often  used  in  the  analysis  of  trends  in 
fertility,  is  a  group  of  women  married  within  a  specified  time 
interval.  The  term  “cohort”  usually  refers  to  a  birth-cohort, 
unless  otherwise  specified. 

'Exploratory  research  in  this  direction  has  been  undertaken 
by  Dr.  Malcolm  S.  Cohen  of  the  Bureau  of  Labor  Statistics. 

'Quite  apart  from  its  use  in  projections,  cohort  analysis  pro¬ 
vides  information  of  great  interest  in  historical  studies,  where  it 
is  inherently  superior  to  period  analysis  in  delineating  the  actual 
historical  experience  of  a  group  of  persons.  Examples  of  studies 


The  techniques  employed  in  short-range  economic 
forecasting  offer  a  wider  range  of  approaches  than  any 
other  area  of  forecasting.  Furthermore,  the  preponder¬ 
ance  of  effort  in  forecastmg,  particularly  in  the  United 
States,  has  been  expended  in  this  field.  Therefore,  a 
review  of  these  techniques,  aimed  at  assessing  their 
potential  applicability  to  long-range  forecasting,  pro¬ 
vides  a  useful  supplement  to  the  techniques  diacosaed 
so  far.®  In  one  of  the  best  critical  summaries  of  these 
techniques,  Roos  distinguishes  five  approaches:  * 
“naive,”  “leading  indexes,”  “comparative  pressures,” 
“opinion  polls,”  and  "econometrics.”  The  naive  tech¬ 
niques  as  outlined  by  Roos  are  essentially  those  that 
have  just  been  summariaed,  with  the  addition  of  cycli¬ 
cal  or  harmonic  analysis  of  deviations  from  trend  and 
the  use  of  autocorrelation  functions.  Cyclical  or  har¬ 
monic  analysis  aims  at  removing  the  effects  of  cyclical 
fluctuations  from  a  given  time  series,  so  that  a  cleaner 
description  of  the  basic  trend  can  be  obtained  for  pur¬ 
poses  or  extrapolation.^®  Autocorrelation  functions  pro¬ 
vide  a  means  of  projecting  a  given  series  by  correlating 
it  with  itself  at  selected  points  in  the  past. 

Without  trying  to  defend  the  use  of  naive  extrapola¬ 
tive  techniques  in  forecasting,  Roos  does  recognize  one 
reason  for  their  persistent  employment,  besides  the 
advantages  of  simplicity  and  speed:  they  turn  out  to  be 
correct  more  often  than  not,  for  the  simple  reason  that 
over  the  shortnin,  persistence  of  a  given  trend  is  more 
common  than  reversals  of  trend.  Hence  the  sophisti¬ 
cated  forecaster  may  well  prefer  the  naive  technique, 
insofar  as  his  performance  is  measured  by  the  ratio 
of  his  successes  to  his  failures.  Unfortunately,  it  is 
precisely  the  reversals  that  most  concern  the  policy¬ 
maker,  and  such  techniques  are  useless  in  forecasting 
reversals. 

The  method  of  leading  indexes  involves  a  search  for 
data  or  some  combination  of  data  in  the  form  of  an 
index  whose  fluctuations  through  time  regularly  pre¬ 
cede  those  of  the  series  to  be  projected.  It  is  not  difficult 
to  provide  examples  of  this  kind  of  precedence  in  the 
realms  of  economic  activity  and  in  population  dy¬ 
namics  as  well.  For  examples,  new  orders  precede  ship¬ 
ment  of  goods,  shipments  precede  sales  and  earnings, 

nsini?  cohort  analvsis  in  both  senses  include  James  M.  Beshers 
(12)  and  Karl  Taeuber  (190).  For  a  critical  comparison  of 
cohort-fertility  and  parity-progression  methods  in  projecting 
fertility,  see  Donald  S.  Akers  (3). 

'Useful  summaries  of  these  techniques  are  provided  in  Crox- 
ton  and  Cowden  (55) ,  and  in  Ferber  and  VerdoOm  (53) . 

'Charles  F.  Roos  (755).  For  an  earlier  discussion  of  extra¬ 
polative  techniques,  see  Albert  G.  Hart  (85) ,  pp.  78-B2. 

"This  is  effected  in  labor  force  projections  by  using  season¬ 
ally  adjusted  data  in  their  development-  - 


an  tqMurge  in  marriagea  precedes  a  rise  in  births  which 
in  turn  precedes  a  rise  in  school  enrollments,  etc.  In 
practice,  however,  such  indexes  are  of  limited  value 
beeain»  the  period  of  the  forecast  is  necessarily  limited 
to  the  length  of  the  timelag  between  the  last  observation 
relating  to  the  leading  index  and  the  corresponding 
successive  observation  on  the  series  being  projected. 
In  general,  the  longer  the  timelag  b^een  the  leading 
index  and  the  variable  being  forecast,  the  less  precise 
is  the  association  between  them.  On  the  other  hand,  if 
the  timelag  is  short,  it  may  not  be  much  longer  than 
the  delay  in  reporting  the  last  observation  on  the  com¬ 
ponents  of  the  leading  index.  In  other  words,  by  the 
time  the  forecaster  has  obtained  the  latest  data  relating 
to  his  leading  index,  much  of  the  period  over  whic  ^  he 
can  safely  pi  eject  may  have  elapsed.  This  technique  is 
useful,  however,  in  the  development  of  target  projec¬ 
tions,  wherein  one  or  more  dq>endent  variables  are 
projected  by  means  of  their  association  with  a  selected 
independent  variable  whose  future  values  have  been 
projected  separately.  For  example,  given  a  hypothetical 
projection  of  the  gross  national  product,  it  is  possible 
to  derive  projections  of  growth  in  the  several  sectors 
of  the  economy. 

The  method  of  comparative  pressures  implies  a  some¬ 
what  greater  theoretical  grasp  of  the  processes  govern¬ 
ing  changes  in  the  series  being  pr^  'ected.  Examples  in 
the  field  of  economic  forecasting,  as  given  by  Roos, 
include  the  use  of  trends  in  the  ratio  of  inventory  to 
sales,  production  to  capacity,  now  orders  to  shipments, 
and  inventory  to  new  orders.^*  Comparable  examples 
in  the  field  of  labor  force  analysis  are  not  easy  to  vis¬ 
ualize.  Trends  in  the  ratio  of  school  enrollments  to  new 
labor  force  entrants  might  be  of  some  value  in  short- 
range  labor  force  projections,  but  the  former  are  laigely 
a  function  of  population  size,  at  least  in  the  United 
States.  Thus,  this  technique  is  not  promising  for  labor 
force  projections  so  long  as  no  simple  stimulus-response 
mechanism  can  be  found  to  rule  over  significant  seg¬ 
ments  of  the  population  of  working  age.^^ 

Hie  hope  that  better  forecasts  might  be  developed 
on  the  basis  of  opinion  polling  has  motivated  their 
recurrent  use,  and  the  accompanying  literature  has  been 

“Roos  op.  cit.,  pp.  379f. 

"The  application  of  the  additional  worker  theory  in  forecast- 
Mng  labor  force  growth  under  stipulated  conditions  of  unemploy¬ 
ment  would  belong  under  this  rubric,  insofar  as  the  growth  of 
the  labor  force  was  predicated  on  some  assumed  ratio  of  addi¬ 
tional  wodters  to  unemployed  workers.  Such  methods  have  had 
only  limited  success  to  date,  because  of  the  difficulty  of  separat¬ 
ing  the  additional  worker  effect  from  a  host  of  other  factors 
which  operate  simultaneously  to  encourage  or  discourage  labor 
force  participation  among  the  more  marginal  workers. 


enlivened  by  considerable  controversy.  The  fundamen¬ 
tal  idea,  of  course,  is  that  the  behavioral  scientist 
should  avail  himself  of  the  unique  opportunity  afforded 
him  by  the  fact  that  those  whose  behavior  he  studies 
are  capable  of  rational  action  and  can  communicate 
their  plans  and  intentions  to  the  observer  or  inter¬ 
viewer.  For  the  forecaster,  the  obvious  course  is  to  se¬ 
lect  a  sample  of  actors  or  agents  in  the  area  of  concern, . 
obtain  from  them  information  as  to  their  intentions, 
and  develop  his  forecast  on  the  basis  of  this  informa¬ 
tion.  The  usefulness  of  this  approach  in  the  area  of 
labor  force  participation  has  yet  to  be  tested,  although 
information  collected  on  reasons  why  persons  are  not 
working  or  looking  for  work  does  provide  some  insight 
into  the  circumstances  under  which  such  individuals 
might  become  active  workers.  While  this  approach 
might  be  of  some  value  in  developing  short-range  fore¬ 
casts,  such  as  forecasting  the  retirement  decisions  of 
older  workers,  it  is  of  doubtful  value  for  purposes  of 
long-range  projection. 

Roos’  criticism  of  opinion  polling  as  a  technique  of 
economic  forecasting  takes  a  different  turn.  He  argues 
that  because  the  expressed  intentions  of  most  business¬ 
men  (and,  a  fortiori,  of  most  consumers)  are  formu¬ 
lated  by  naive  extrapolations  of  trends  and  are  greatly 
influenced  by  strong  bandwagon  effects,  any  sampling 
of  these  intentions  is  likely  to  yield  results  closely 
similar  to  those  obtained  by  naive  extrapolation.  In 
addition,  the  strong  dominance  of  trends  observed  over 
the  period  immediately  preceding  the  time  at  which 
the  forecast  is  prepared  makes  it  highly  unlikely  that 
this  technique  could  reveal  a  turning  point  in  advance 
of  its  occurrence.  On  the  other  hand,  where  a  band¬ 
wagon  effect  becomes  dominant  over  current  trends, 
the  resultant  forecast  is  quite*  likely  to  predict  a  false 
reversal.*® 

The  experience  gained  in  surveys  of  establishments 
whereby  information  is  obtained  on  anticipated  em¬ 
ployment  needs  casts  further  doubt  on  the  utility  of  this 
technique  as  a  means  of  developing  reliable  forecasts. 
In  his  study  of  the  employment  forecast  survey  in 
Canada,  Hartle  concludes  that  these  surveys  cannot  be 
expected  to  provide  a  suitable  basis  for  reliable  fore¬ 
casts  of  employment.**  His  findings  reveal  that  a  num¬ 
ber  of  ingenious  efforts  to  improve  the  forecasting 
reliability  of  the  data  obtained  in  these  surveys  proved 
unavailing.  For  example,  efforts  aimed  at  eliciting  more 

"Roos  op.  cit.,  pp.  381-386.  The  central  problem  here,  la 
with  most  opinion  polling  techniques,  is  that  expert  or  carefully 
lormulated  views  tend  to  be  submerged  in  an  averaging  procew 
which  is  heavily  weighted  by  the  larfjer  mass  of  inexpert  vpinion. 

"Douglas  G.  Hartle  (86),  pp.  93  and  lOSff. 
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careful  responses  were  not  successful.  Most  schedules 
are  filled  out  hastily,  and  often  by  persons  well  below 
the  decisionmaking  levels  in  their  organizations.  Also 
unsuccessful  were  attempts  to  adjust  the  reqtonses  to 
take  account  of  ait  apparent  seasonal  factor  in  the  re¬ 
ported  intentions  of  the  establishments  surveyed. 
Hartle’s  analysis  of  the  individual  forecasts  of  par¬ 
ticular  establishments  showed  that  they  were  actually 
less  accurate,  on  the  whole,  than  they  would  have  been 
had  they  been  derived  by  simple  extrapolation  of  past 
trends.  Furthermore,  these  forecasts  were  characterized 
by  the  same  regressive  feature  noted  in  an  earlier  study 
by  Ferber — i.e.,  a  pronounced  tendency  to  forecast 
reversals  of  trend  in  employment.*®  Hartle  is  unwilling 
to  end  his  analysis  on  a  negative  note,  and  suggests 
instead  that  information  on  employers’  forecasts  of 
employment  might  be  incorporated  into  a  larger  model, 
rather  than  being  used  to  project  employment  directly. 

The  use  of  econometric  techniques  in  the  preparation 
of  projections,  forecasts,  and  predictions  is  a  very  large 
topic  in  itself.  The  term  “econometric”  refers  to  a  com¬ 
bination  of  economic  theory,  quantitative  economic 
data,  and  mathematical-statistical  tools  of  analysis. 
This  definition  at  once  suggests  certain  limitations  in 
the  applicability  of  econometrics  to  labor  force  analysis 
or  labor  force  projections,  insofar  as  the  labor  force 
activities  or  proclivities  of  a  population  cannot  be  fully 
subsumed  under  a  scheme  of  economic  determinism. 
However,  it  does  have  great  potential  for  fruitful  appli¬ 
cation  in  labor  force  analysis  inasmuch  as  it  becomes 
possible  to  incorporate  measures  of  appropriate  be¬ 
havioristic  variables  into  the  econometric  equations. 

Broadly  conceived,  econometric  techniques  comprise 
an  even  wider  spectnim  of  approaches  than  those  dis¬ 
cussed  up  to  now,  ranging  from  simple  regressions  to 
highly  complex  models  and  model  systems.  Econometric 
techniques  cannot  therefore  be  distinguished  by  reason 
of  complexity.  What  characterizes  these  techniques  is 
the  systematic  integration  of  quantitatively  expressed 
economic  theory,  the  methods  of  mathematical  statistics, 
and  data  in  the  form  of  quantitative  measures  or 
indices  of  economic  phenomena.  TTie  theory  is  the 
source  of  the  models  that  are  constructed;  the  data 

“Hartle’s  study  revealed  that  over  a  3-year  period,  388  manu¬ 
facturing  establishments  experienced  about  1,700  individual 
turning  points,  while  predicting  over  3,000  of  them.  (Hartle, 
Ibid.,  p.  102.)  The  earlier  study  cited  by  Hartle  is  Robert 
Ferber,  “The  Railroad  Shippers’  Forecasts”  (1953).  An  excel¬ 
lent  collection  of  articles  on  the  uses  and  limitations  of  data  on 
expectations  is  Mary  Jean  Bowman,  ed.  (15).  More  recent 
studies  include  Franco  Modigliani  and  Kalman  J.  (]ohen  (1.33) 
and  Andrew  S.  Ozga  (145).  For  a  careful  review  of  the  pre¬ 
dictive  value  of  consumers’  attitude  surveys,  see  Eva  Mueller 
(140). 


provide  their  content  and  thereby  provide  the  cromal 
linkage  between  theory  and  empirical  reality;  the  ata- 
tistical  techniques  permit  the  expression  of  the  postu¬ 
lated  relationships  in  testable  form — typically  as  equa¬ 
tions  to  be  solved.  A  basic  requirement  in  all 
econometric  procedures  is  the  detailed  explication  of 
the  assumptions  and  procedures  adopted  in  a  given 
operation.  Hence  the  strong  reliance  upon  quantitative 
expressions  and  measurements  of  relevant  variables.** 

Insofar  as  forecasting  or  prediction  is  concerned,  the 
basic  requirements  of  the  econometric  approach  are 
readily  set  forth.  The  difficulties  commonly  encountered 
in  trying  to  fulfill  these  requirements  are  not  so  easily 
summarized.**  According  to  Christ,  the  equations  used 
in  econometric  analysis  should  have  the  following  prop¬ 
erties:  relevance,  simplicity,  theoretical  consistency,  ex¬ 
planatory  power,  accurate  coefficients,  and  forecasting 
or  predictive  ability.*®  Relevance  means  that  these  equa¬ 
tions  must  have  a  direct  bearing  on  a  significant  prob¬ 
lem.  Simplicity  means  workability — ^i.e.,  the  equation 
system  must  be  amenable  to  manipulation  and  must  be 
comprehensible.  Theoretical  consistency  means  that  the 
relations  postulated  by  the  theory  must  not  be  contra¬ 
dicted  by  those  expressed  in  the  equations.  Explanatory 
power  means  relevance  to  actual  phenomena,  as  re¬ 
flected  in  available  data.  Accuracy  of  coefficients  means 
measurability  within  a  finite  range  of  variability. 
Finally,  predictive  ability  means  the  ability  to  generate 
accurate  forecasts  with  the  given  equations.  Christ 
makes  it  perfectly  clear  that  the  latter  objective  cannot 
be  attained  unless  the  former  requirements  are  met. 

The  implications  of  these  requirements  for  the  prob¬ 
lem  of  projection  can  be  brought  into  focus  in  terms 
of  Christ’s  useful  distinction  between  forecasts  (pre¬ 
dictions)  designed  to  guide  the  policymaker  and  those 
designed  to  test  hypotheses.**  A  forecast  is  most  fruitful 

“An  elementary  introduition  to  econometric  techniques  is 
Michael  J.  Brennan  (18).  Three  works  which  place  peuticular 
emphasis  on  the  practical  techniques  of  economic  forecasting 
are  Henri  Theil  (194,  195)  and  Rohert  G.  Brown  (21).  A  re¬ 
cent,  somewhat  advanced  presentation  of  the  statistical  methods 
being  developed  to  cope  with  econometric  problems  is  E.  Malin- 
vaud  (117).  An  excellent  and  somewhat  mote  general  treat¬ 
ment  of  econometrics  is  Carl  F.  Christ  (26). 

"As  noted  earlier,  most  econometricians  use  the  terms  “fore¬ 
cast”  and  “prediction”  synonymously.  Both  Christ  and  Theil 
use  either  term  in  relation  to  statements  about  behavior  which 
has  not  yet  been  observed  (although  it  may  have  occurred  in  the 
past).  In  the  English  translation  of  his  work,  Malinvaud  uses 
the  term  “prediction”  exclusively. 

“Christ  op.  Lit.,  pp.  4ff. 

“Christ,  Ibid.,  p.  293.  Christ  repeats  Pigou’s  apt  phraseology 
here,  distinguishing  Letween  forecasts  which  are  designed  to 
yield  fruit  and  those  which  aim  to  yield  light.  Compare  Knowles 
(99),  p. 7. 


to  Uw  policymaker  if  it  ia  conditional  upon  variables 
whose  values  reflect  altemative  policies  or  wbicb  can 
hi  cAwted  in  predetermined  ways  by  such  policies. 
Id^y,  such  forecasts,  granted  the  iniBvitable  ceteris 
paiAus  caveat,  will  tdl  the  policymaker  that  the  adop> 
tion  of  policy  A  is  likely  to  produce  condition  X,  B 
vdB  yidd  Y,  etc.  These  forecasts  must  also  extend  far 
enough  into  the  future  to  aUow  the  consequences  of 
given  policies  to  work  themselves  out,  and  to  allow  the 
policymaker  some  freedom  of  decision. 

The  needs  of  enlightenment  in  hypothesis  testing  are 
flomewhat  less  restrictive,  since  forecasts  which  extend 
only  from  an  earlier  past  period  to  one  in  the  more 
recent  past  frequently  afford  highly  useful  tests  of 
qiecific  hypotheses  as  well  as  complete  models.  Never¬ 
theless,  confirmation  or  disconfirmation  of  an  hypo¬ 
thesis  in  the  light  of  purely  historical  data  is  seldom 
entirely  conclusive  with  regard  to  its  possible  status 
in  the  future,  at  least  in  the  behavioral  sciences. 

With  the  above  uses  of  forecasts  in  mind,  we  can 
now  summarise  the  problems  encountered  in  satisfying 
the  six  requirements  for  econometric  equations  as  set 
forth  by  Christ.  First,  the  requirement  of  relevance 
imposes  the  severe  restriction  that  the  zeal  to  simplify 
matters  by  means  of  arbitrary  classification  and  arti¬ 
ficial  quantification  must  be  tempered  by  a  willingness 
to  struggle  with  the  complexities  of  the  real  world. 
Highly  abstract  models  provide  neither  guidance  to 
the  poBcymaker  nor  teats  for  the  hypotheses  of  the 
researcher.  In  his  highly  critical  exposition  of  economic 
methodology,  Schoeffler  questions  the  relevance  of  what 
he  terms  “mechanical  behavior  models”  on  the  grounds 
that  thej'  necessarily  assume  fixed  patterns  of  response 
to  given  sets  of  environmental  factors.  He  argues  that 
in  its  actual  operation,  a  firm  (for  example)  may  or 
may  not  act  to  maximize  its  profits,  minimize  rides,  etc. 
Thus  any  model  which  assumes  a  necessary  rather  than 
voluntary  response  to  given  conditions  is  aitificially 
mechanistic.^ 

The  simplicity  requirement  must  be  considered  at 
two  levels — that  of  the  technician  and  that  of  the  user. 
For  the  former  group,  simplicity  is  largely  a  matter  of 
explicitness  of  assumptions  and  sufficient  clarity  and 

"Sidney  Schoeffler  (idi),  pp.  18f{.  The  employment  of  oto- 
ehasUc  models  wherein  patterns  of  response  are  6xed  only  in  a 
prababilistic  senw  would  appear  to  overcome  this  objection. 
See,  for  example,  the  admittedly  grandnise  Markovian  model  for 
the  analysis  (only  in  principle!)  of  historical  processes,  as  out¬ 
lined  by  Fianklin  M.  Fisher  (54) .  Earlier  descriptions  of  the 
general  nature  of  stochastic  models  in  economics  include  Trygve 
Haavelmo  (90)  and  Jacob  Manchak  (223).  Maischak's  article 
is  reproduced  in  Schoeffler's  book.  Schoeffler’s  criticism  of  the 
probability  approach  appears  in  Schoeffler,  Ibid.,  ppw  124B. 


detsil  in  the  explanation  of  one’s  procedure — ^require¬ 
ments  which  are  fully  satisfied  if  and  only  if  the  model 
can  be  described  by  means  of  equations,  inequalities, 
or  similar  mathematical  constructs.  The  advent  of  com¬ 
puters  and  their  rapid  improvement  has  greatly  reduced 
the  restrictiveness  of  another  simplicity  requirement, 
that  of  computational  facility.  Nevertheless,  the  finite 
capacity  of  existing  computers  shall  always  constitute 
an  ultimate  limitation  on  the  complexity  of  models  or 
equation  systems  which  can  be  handled  effectively. 
To  the  users,  particularly  those  lacking  in  technical 
training,  the  clearest  econometric  model  is  typically 
the  most  opaque,  and  the  greater  the  mass  of  quantita¬ 
tive  detail  offered,  the  less  intelligible  the  totality  be¬ 
comes.  The  common  reaction  of  the  technician  to  this 
dilemma  is  inadequate:  He  would  merely  insist  that 
persons  who  use  the  results  of  liis  work  be  equipped 
to  understand  it  in  his  terms.  But  this  view  either 
underestimates  the  intellectual  achievement  of  the  tech¬ 
nician,  or  overestimates  the  time,  energy,  and  possibly 
even  the  capacity  of  the  user.  A  more  promising  alter¬ 
native,  and  one  which  often  yields  beneficial  side 
effects,  is  to  append  a  nontechnical  or  verbal  account 
of  the  work.  This  task  requires  considerable  skill  in  its 
own  right,  since  it  is  necessary  to  explain  technical 
matters  without  distorting  them  by  oversimpufication, 
and  yet  without  writing  a  basic  textbook  for  the  unini¬ 
tiated.*^  The  beneficial  side  effects  are  readily  apparent 
in  Baumol’s  apt  distinction  between  explicit  premises 
or  assumptions  and  transparent  ones.  TTie  great  power 
of  mathematical  expressions  lies  in  their  explicitness 
and  economy.  However,  the  implications  of  these  terms, 
when  they  purport  to  relate  to  the  real  world  via  the 
equations  of  a  given  model,  are  often  hidden  from  view. 
Thus  their  interpretation  in  ordinary  language  may 
often  bring  to  light  the  full  implications  of  a  given  set 
of  assumptions  and  procedures  in  a  way  which  cannot 
be  accomplished  by  a  purely  mathematical  statement.** 

**A  small  but  growing  number  of  econometricians  seem  to  be 
endowed  with  sufficient  patience  and  expository  skill  to  reveal 
the  workings  of  their  models  in  readable  English.  See,  for  ex¬ 
ample,  (Thrist,  op.  cU.,  Oopper  Almon,  Jr.  (4) ,  Daniel  B.  Suits 
(187),  and  Jack  Alterman  (5). 

"William  J.  Baumol  (8) ,  p.  94ff.  One  of  Baumol’s  examples 
relates  to  the  assertion  that  the  supply  or  demand  function,  ex- 
pressed  in  terms  of  price  only,  is  homogenous  of  degree  zero. 
As  be  points  out,  this  assertion  is  persuasive  in  that  it  seems  to 
argue  only  that  the  unit  of  currency  chosen  is  of  no  economic 
significance.  However,  its  uncritical  adoption  comes  to  grief 
when  it  is  recognized  that  it  implies  that  a  doubling  of  all  price 
quotatioiis  would  not  affect  decisions  even  if  the  number  of  units 
of  currency  failed  to  double  also.  Only  a  fuller  explanation  of 
the  assumption  in  question  rather  than  merely  listing  the  appro¬ 
priate  function  as  an  element  in  the  model,  would  avoid  this 
kind  of  difficulty. 
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In  short,  the  demand  for  greater  explicitness,  which  is 
the  road  to  simplicity  for  the  technician,  must  he 
matched  by  a  demand  for  transparency  whereby  the 
user  can  assess  the  true  implications  of  the  assump¬ 
tions  which  are  so  neatly  expressed  in  the  form  of 
equations. 

This  leads  us  to  the  requirement  of  theoretical  plausi¬ 
bility.  As  Crossman  puts  it  so  well,  a  theory  which  can 
provide  a  basis  for  adequate  prediction  or  forecasting 
must  not  only  explain  the  static  relationships  among 
the  relevant  factors,  but  must  also  provide  a  quantita¬ 
tive  description  of  their  dynamic  interaction  and  change 
through  time.  This  is  indeed  a  tall  order.®*  The  mani¬ 
fold  difficulties  to  be  overcome  before  such  a  theory, 
can  be  developed  are  brought  out  in  Schoeffler’s  critique 
of  macroeconomic  models.  Schoeffler  stresses  the  weak¬ 
nesses  of  their  theoretical  underpinning,  particularly 
in  reference  to  their  use  in  forecasting  and  prediction. 
He  points  out  that  the  need  for  manageable  equations 
reduces  the  number  of  variables  whose  effects  can  be 
determined,  so  that  many  influences  are  ignored  or 
subsumed  under  a  residual  “error”  term.  Furthermore, 
the  parameters  in  the  given  equations  necessarily  re¬ 
flect  past  influences  only;  there  can  be  no  assurance 
that  these  will  remain  fixed  in  future  time.  Most  dam¬ 
aging,  however,  is  the  fact  that  the  equations  reveal 
nothing  concerning  the  chains  of  causation  which 
underly  the  statistical  estimates  that  are  obtained.  In 
other  words,  the  macroeconomic  approach  still  involves 
the  manipulation  of  a  black  box  whose  inner  workings 
remain  hidden  from  view.  Schoeffler  concludes  with  an 
expression  of  qualified  pessimism.  While  he  recognizes 
that  any  judgment  concerning  the  future  development 
of  econometric  models  is  premature,  he  argues  that 
in  their  present  form,  they  cannot  be  expected  to  pro¬ 
vide  dependable  forecasts.** 

**£.  R.  F,  W.  Crossman  {33,  34).  The  latter  report  provides  a 
more  detailed  development  of  his  thesis. 

“Schoeffler  op  cit.,  p.  116f{  and  130ff.  Of  course,  no  other 
technique  can  do  better,  unless  by  accident.  Comparisons  of 
forecasts  made  judgmentally  with  those  produced  with  econo¬ 
metric  models  are  generally  inconclusive,  partly  because  these 
techniques  are  not  really  as  distinct  as  they  appear  in  formal 
descriptions.  On  this  point,  see  George  Jasii  et  al.  (9d).  In 
actual  practice,  the  judgmental  forecaster  typically  uses  what¬ 
ever  quantitative  data  and  techniques  he  deems  appropriate, 
while  the  actual  work  of  the  model  builder  is  infused  with  judg¬ 
mental  decisions  at  every  step.  For  an  enlightening  comparison 
of  the  more  "judgmental”  British  economic  forecasting  tech¬ 
niques  with  the  more  rigorous  Dutch  techniques,  see  R.  L 
Marris  (123).  Both  Marris  and  Jaaai  and  associates  reeogniae 
the  superiority  of  the  more  rigorous  econometric  techniques  "in 
principle  and  in  the  long  run,”  while  admitting  their  practical 
limitationa  at  present 


The  inability  of  existing  theory  to  fulfill  the  demand 
for  causal  understanding  poses  a  dilemma  for  the  fore¬ 
caster  whose  insistence  upon  methodological  rigor  ont- 
weighs  the  felt  need  for  immediate  results.  For  those 
with  a  more  pragmatic  orientation,  however,  the  di¬ 
lemma  is  avoided  by  recognizing  that  reliable  forecasts 
can  often  be  made  on  the  basis  of  perceived  regulaii- 
ties  in  the  absence  of  a  complete  theoretical  understand¬ 
ing  of  the  underlying  mechanisms.  In  other  words, 
reliance  on  simple  techniques  of  extrapolation  will  often 
yield  good  forecasts.  Unfortunately,  as  we  have  already 
noted,  such  procedures  assume  structural  stability  and 
provide  no  basis  for  determining  either  the  suitability 
of  that  assumption  or  the  likelihood  that  a  structural 
change  will  occur.®*  This  problem  is  less  serious  in 
relation  to  hypothesis  testing  than  in  relation  to  fore¬ 
casting,  since  hypotheses  can  often  be  tested  over  past 
periods  known  to  be  structurally  stable.  In  practice, 
however,  our  inability  to  discover  long  periods  of  struc¬ 
tural  stability  in  the  past  hardly  supports  the  conven¬ 
ient  assumption  of  such  stability  in  the  future. 

The  requirement  of  explanatory  ability  amounts  to 
the  assertion  that  an  equation  system  cannot  be  prop¬ 
erly  designated  “econometric”  unless  it  correlates  with 
empirical  reality  according  to  certain  rules  of  con¬ 
gruence.  The  internal,  logical  consistency  of  such  a 
system  is  a  necessary,  but  not  a  sufficient  condition  for 
its  designation  as  a  model.  A  further  implication  of 
this  requirement  is  that  the  data  to  which  the  model 
refers  must  have  a  wider  domain  than  that  of  the 
model’s  definitional  properties;  otherwise,  the  model 
can  have  no  utility  except  as  a  descriptive  sununary  of 
its  internal  structure.  Obviously,  the  usefulness  of  any 
econometric  model  in  forecasting  is  largely  dependent 
upon  its  explanatory  power.  What  is  perhaps  less 
obvious  is  that  a  model  lacking  significant  explanatory 
power  cannot  be  used  to  test  hypotheses  either,  wee 
such  a  test  necessarily  requires  some  extension  beyond 
the  model’s  definitional  properties.** 

The  need  for  accurate  coefficients,  as  described  by 
Christ,  is  equivalent  to  the  familiar  statistical  require¬ 
ment  of  significant  associations  or  correlations.  It  can 
only  be  met  by  the  discovery  of  explanatory  (indepen¬ 
dent)  variables  whose  estimated  (nonzero)  coefficients 

“The  underlying  structure  mutt  be  putiallT  known,  of 
course;  otherwise,  the  notion  of  structural  change  would  bo 
meaningless.  For  a  more  thorough  exposition  of  this  iaane,  tea 
Marschak  (134). 

“A  recent  summary  of  the  relations  between  theory  and  data 
in  science  it  that  of  Henry  Margenau  {131).  Margenau  baa  ez- 
pretsed  these  principles  in  relation  to  a  variety  of  bohavorial 
tcieuoes  over  the  years.  Also  useful  in  this  regard  is  tbe  ozod- 
lent  article  hy  Fritz  Machlnp  {116). 
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lum  MMptiiblr  low  Mtimated  aUndard  deviatiooa.  The 
■Mmb  for  onoh  variablee  io  of  cooroe  a  ooitral  concern 
of  aB  HmUed  ackotifio  leaearch.  Unfortunately,  none 
of  tio  pieoeiibod  reaeaieh  practioea  can  guarantee  the 
diaeovery  of  independent  variablea  displaying  the 
aon^-for  tegoIarUea  in  aaaociation  with  die  given 
dependent  variable.  Even  when  atatiatically  dgnificant 
and  anfBdently  preeiae  coefficienta  are  obtained,  the 
qneation  remaine  as  to  their  economic  significance.  In 
terms  familiar  to  social  scientists,  we  can  say  that 
reliability  of  a  measure  does  not  imply  its  validity. 
The  latter,  like  the  question  of  economic  significance, 
can  only  be  determined  on  the  basis  of  a  broad  theoreti¬ 
cal  and  practical  understanding  of  the  subject  area. 

Ibe  last  of  Qirist’s  catalogue  of  objectives  for  econo¬ 
metrics  is  the  requirement  of  forecasting  ability  itself — 
an  objective  which,  as  we  have  noted,  implies  the  others 
and  is  an  ultimate  test  of  their  achievement.  The  ability 
to  forecast  within  tolerable  limits  of  error  is  essential 
not  only  to  provide  some  notion  of  probable  trends  or 
outcomes  of  current  decisions,  but  also  to  teat  hypo¬ 
theses  by  comparing  the  model’s  prediction  (reflecting 
a  specified  hypothesis)  with  the  actual  outcome.  In 
short,  some  forecasting,  ability  is  essential  if  econo¬ 
metric  techniques  are  to  be  useful  either  in  policy¬ 
making  or  in  the  testing  of  hypotheses.”  Here  also, 
Schoeflkr’s  comments  express  the  current  situation 
with  raqiect  to  forecasting  ability  in  blunt  terms.  He 
observes  the  familiar  classification  of  econometric 
equations  into  four  groups  (technological,  bdiavioral, 
legal,  and  definitional)  and  notes  that  stability  over 
time,  which  is  crucial  in  determining  the  reliability 
of  a  forecast,  cannot  be  guaranteed  for  any  of  these, 
save  for  the  definitional  group,  which  is  arbitrary  to 
begin  with.” 

"At  nwil  poiau  oat,  the  problem  of  determining  the  nccu- 
racy  of  »  forecaet  most  be  dietingubhed  from  that  of  verifying 
the  forecnaiing  procedure  itKlf.  The  latter  problem  can  bf 
handled,  at  Iscit  in  the  cate  of  point  predictiona,  by  testing  the 
Ball  hypotheaia  that  the  obienred  prediction  errors  are  members 
of  a  parent  population  whoae  errors  have  zero  meant,  etc.  See 
Henri  Theil  (193),  p.  30.  In  their  ditcustion  of  Theil's  paper 
(Ibid,,  p.  47\  Lehman  and  Knowles  stress  the  further  distinc¬ 
tion  between  the  aocurney  of  a  projection  ond  its  usefulness  in 
serving  the  purpose  for  which  it  was  prepared. 

"Schoefler  op.  eit,  p.  1301.  Schoefler’s  strictures  lose  some 
of  their  force  when  applied  to  current  eforts  at  model  construc- 
liM^  srhere  greater  atteatioa  it  paid  to  the  more  limited  infer, 
aneea  sddeh  can  be  drawn  under  conditioas  srhich  only  ap- 
pradmata  the  theoretical  teuuiremenls  of  the  given  model.  See, 
for  eaam^  Albert  Ando  et  aL  (6).  For  Schoefler't  detailed 
scbaam  tor  tdeB'’llc  predktioa  la  economies,  see  cha  4  aad  0 
of  his  study. 

Jemend's  comments  are  pertinent  ia  this  regard.  In  tuat- 
asariiiBg  IVeil't  analyids  of  the  Dutch  experience  with  econo- 


The  sdveast  of  sutomatic  data  processing  via  com¬ 
puters  has  given  rise  to  another  promising  projective 
technique — the  development  of  projections  by  means 
of  computer  simulation.  The  essential  idea  is  clearly 
described  by  Orcutt  and  associates  as  the  construction 
of  bdiavioral  (  or  econometric)  models  whose  dements 
comprise  data  on  the  observed  behavior  of  a  sample  of 
individuals  or  housdiold  units  (or  firms).”  Gtnven- 
tionally,  the  elements  of  a  model  are  integrated  accord- 
ing  to  the  postulates  of  a  given  theory  and  the  hypo¬ 
theses  derived  therefrom.  Since  the  model  is  designed 
to  reflect  the  patterns  of  interaction  postulated  by  the 
underlying  tliMry,  it  can  be  uicd  to  predict  the  out¬ 
come  of  qtecified  hypotheses  derived  therefrom.  By 
using  a  microanalytic  approach,  the  model  can  be  con¬ 
structed  to  reflect  actual  behavior  of  the  microunits 
(individuals,  households,  firms,  etc.)  as  they  interact 
over  time,  rather  than  being  confined  to  the  dictates  of 
a  theory  that  has  been  adopted  a-priori.  Once  such  a 
behavioral  model  is  operative,  it  can  be  used  to  simulate 
the  probable  responses  of  outcomes  reflecting  a  variety 
of  specified  initial  conditions,  thus,  in  effect,  testing  a 
variety  of  hypotheses  or  a  range  of  predictions. 

The  basic  requirement  for  the  construction  of  such 
dynamic  models  (given  the  requisite  data  processing 
facilities)  is  the  availability  of  disaggregated  data  of 
sufficient  quantity  and  quality  to  provide  a  series  of 
comparable  observations  through  time.  As  might  be 
expected,  the  construction  of  a  dynamic  model  (dy¬ 
namic  because  it  incorporates  changes  in  its  constituent 
elements  occurring  through  time)  involves  some  trade¬ 
off  between  the  desire  to  maximise  fidelity  to  the 
complexities  of  the  real  world  as  observed  and  the 
need  to  retain  a  model  of  sufficient  simplicity  to  be 
manipulable.  Greenberger’s  recommendations  for  han¬ 
dling  this  problem  are  threefold ;  First,  the  construction 
of  the  model  requires  the  services  of  a  subject-matter 
specialist  working  in  concert  vrith  a  computer  pro- 
gramer.  Unless  the  mode)  to  be  used  in  simulation  takes 
account  of  whatever  peculiarities  the  data  may  contain, 
the  resuhant  simulation  is  unlikely  to  be  sufficiently 
realistic  to  have  much  practical  value.  The  subject- 

metric  forecasts,  Joureoc!  indicale*  that  ihe  bulk  of  the  fore- 
cutinf  erroiv  dUappeared  wben  actual  raluet  for  the  model't 
esofcaoua  variaUea  were  MbslitBted  for  the  projected  valu^ 
oriaivtlhr  need  ia  forecaMiaf.  Jouveael  cooclndea,  therefore, 
that  the  proUeaa  of  s  casiog  the  proper  raluea  to  iuaert  la  the 
forrcaal^ag  nwdrJ  remaina  a  ••-riou.  limitttioa  in  the  uie  of  tuch 
modcla  for  loog-ranta  ptojectioca.  See  Bertraad  de  JouTeael 
(95).  pp.  I94i 

"Coy  H.  Orcutt  cl  oL  (143),  pp.  8-11  A  woie  recent  aam- 
mary  of  cuirvai  treads  aad  prohiema  ia  cocapawr  sUauUliea  b 
the  behavioial  Kiesces  it  Jamca  M.  Beahere  (ed.),  (13). 


matter  specialist  alone  can  provide  the  necessary  infor¬ 
mation  as  to  the  peculiarities  of  the  data  to  hc'  used  as 
inputs.  Second,  the  model  should  be  constructed  incre¬ 
mentally,  so  that  comparisons  between  the  model  and 
the  data  can  be  made  at  each  stage,  and  new  data  be 
incorporated  as  they  become  available.  Third,  there  is 
need  fer  an  on-line  programing  system  associated, 
ideally,  with  a  multiaccess  computer  installation,  so 
that  the  researcher  can  enjoy  easy  and  frequent  access 
to  the  computer.*® 

The  outlook  for  the  satisfaction  of  these  three  re¬ 
quirements  is  highly  favorable  at  the  present  time.  The 
development  of  Fortran  and  similar  computer  lan¬ 
guages,  together  with  the  rapid  proliferation  of 
“canned”  programs,  greatly  facilitates  the  use  of  auto¬ 
matic  data  processing  machinery  by  subject-matter  ex¬ 
perts.  The  major  obstacle  is  the  unavailability,  at  least 
in  convenient  form,  of  microdata  (i.e.,  disaggregated 
data)  in  sufficient  quantity  for  large-scale  analyses.^' 
If  such  data  as  are  available  prove  to  be  adequate  for 
this  purpose,  and  if  future  data-gathering  processes  can 
be  readily  geared  to  the  needs  of  microanalysis,  com¬ 
puter  simulation  techniques  offer  great  promise  as  a 
means  for  generating  a  wide  variety  of  projections  re¬ 
flecting  many  different  combinations  of  specified  con¬ 
ditions  and  assumptions.  Of  even  greater  promise  is 
the  possibility,  through  computer  simulation,  of  ex¬ 
perimenting  with  changes  in  a  variety  of  social  and 
economic  factors  under  conditions  which  approximate 
the  potential  impact  of  alternative  policy  decisions  in 
the  real  world.**  The  inability  of  social  scientists  to 
carry  out  meaningful  experiments  on  large-scale  social 
systems  has  long  been  recogniied  as  one  of  the  funda¬ 
mental  obstacles  to  the  development  of  social  science. 
While  no  model  can  ever  be  a  fully  adequate  substitute 
for  the  reality  it  represents,  the  patient  accumulat’on 
of  the  results  of  experimentation  with  a  variety  of 
models,  each  of  which  captures  a  selected  aspect  of 
social  reality,  may  eventually  yield  far  more  reliable 
information  regarding  the  processes  of  change  and 


interaction  than  can  be  hoped  for  through  the  analyaia 
of  aggregated  data  alone.  In  short,  etqwrimentation 
with  models  via  computer  simulation  may  represent  a 
useful  substitute  where  eiperimentation  in  the  social 
system  itself  is  impossible, 

A  further  stage  of  possible  devdopment  is  already 
evident  in  the  on-going  research  of  Richard  Stone  and 
associates.  Stone’s  efforts  are  directed  toward  the  de¬ 
velopment  of  a  system  or  hierarchy  of  econometric 
models  representing,  in  their  totality,  the  functioning 
of  the  British  economy.  As  this  system  is  devdoped, 
each  submodel  is  designed  to  reflect  the  operation  of 
some  major  sector  of  the  economy.  These  snbmodds 
are  linked  together  so  that  thdr  exogenous  portions 
acquire  input  from  the  composite  modd,  and,  in  turn, 
provide  output  which  modifies  the  exogenous  portion 
of  the  main  model  at  a  later  stage.  One  advantage  of 
working  with  a  number  of  submodels  is  that  each  of 
them  can  be  kept  at  a  manageable  levd  of  complexity. 
Another  advantage  is  that  subject-matter  experts  can 
be  utilized  most  effectively  in  the  construction  of  sub¬ 
models  relating  to  their  particular  speciahies.**  We 
shall  return  briefly  to  the  far-reaching  implications  of 
this  kind  of  long-range  research  in  the  concluding 
chapter. 

The  preceding  discussion  of  techniqnes  of  projection 
has  dealt  exclusively  with  quantitative  projections — at 
least,  with  projections  which  conk)  be  expressed  quan¬ 
titatively,  even  if  arrived  at  judgmentally.  To  complete 
the  discussion,  it  is  necessary  to  consider  one  more 
technique,  one  which  is  designed  to  yield  reasoned 
judgments  concerning  selected  features  of  a  possible 
future  in  qualitative  terms.  This  approach  has  been 
termed  the  “Delphi  technique,”  recalling  the  Delphic 
oracles.  The  basic  purpose  of  the  Delphi  technique  is 
to  express  a  systematic  pooling  of  expert  opinion  with 
respect  to  prospective  outcomes  of  current  trends  and 
developments  in  a  particular  subject  area.**  The  pro¬ 
cedure  ii.volves,  first,  the  prqsaration  of  a  question¬ 
naire  relating  to  the  subject  area  of  concern.  The 


“Martin  Creenberser  (M),  pp,  148  and  ISSf. 

"J.  E.  Morton'i  recommendalion*  are  releTani  in  thi»  regard. 
See  J.  E.  Morton  (138). 

“■Tile  adrantagei  of  rompuler  .!  nulalion  in  a  rariely  of  ex- 
perimenl.t!  reaearch  projecit  are  aummarired  by  Richard  E. 
Dawaon  (.19)  and  Guy  H.  Crrull  (144).  Alao  see  Diana  Crane 
(31).  Crane's  article  includes  a  uarful  bibliofraphy  on  ibe  tab- 
jecl.  Suffealire  indications  of  the  direclinna  which  might  be 
taken  by  computer  aimjialion  of  aocioeconnmic  brharior  are 
contained  in  Herbert  .A.  Simon  (ITS,  IT6) .  .A  more  technical 
and  very  general  anmmary  of  the  potential  uara  of  simulation  is 
•And." w  .S.  Scott  (162).  For  a  report  of  spreifir  experience  with 
such  aimnlalkra  in  demography,  are  Jeanne  Claire  Ridley  and 
Mindel  C  Shepa  (JS2). 


“Richard  N.  Stone  (182),  p.  Ml.  Prriimiatry  reaulta  of  an 
rxerriae  designed  to  yield  eatimalea  of  the  anpply  and  demand 
for  labor  in  1970  are  given  in  Richard  N.  Stone  and  Alan  Bton 
(183),  p  741.  For  a  brief  accounl  of  the  anticipaled  ben^a  ol 
an  economic  model  ayslem  for  pnrpaaea  of  aacial  and  aeonomic 
planning  in  Britain,  arc  Stono  (l$4). 

“T  J.  Gordon  and  Oiaf  Kefaoer  (63).  In  thia  islereating 
study,  the  authors  report  tb-  .-csulta  ohtainod  via  iht  Delphi 
trehnitfur  on  a  wide  range  of  anhjecta,  inrinding  aach  dieerae 
matters  at  anticipated  popolatinn  growth,  the  pmiparta  br 
pooce.  anticipated  treapona  development.  aaSidpted  ptugam  in 
•pace  eiploration,  and  predictona  ol  akeanihe  hrenkthiongha  in 
a  number  of  helds.  For  an  rxoriknt  anmanary  of  thia  lechniuM 
and  its  limitations,  ace  Daniel  Bell  (tO). 
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plmdiig  of  the  heme  on  the  qaeetioniieire  mey  Gonaiat 
of  eay  combinetKm  of  atetementa  with  which  panel 
of  oqieita  can  eapreae  agreement  or  diaagreement,  or 
qneitioM  which  t^  can  anawer  by  aelecting  the  moat 
qi|Nro]Mriate  of  the  reaponaes  provided,  u  in  a  multiple* 
dwice  examination.  The  re^mndenta  may  alao  be  per¬ 
mitted  to  write  in  their  viewa  or  anawera  to  q>ecific 
qoeationa.  Next,  an  qtpropriate  panel  of  experu  in  the 
paiticular  area  of  conoem  moat  be  aekcted.  When  the 
paad  memhera  have  received,  filled,  and  returned  their 
qoeationnairea,  the  fiiat  phaae  of  the  operation  haa 
begun.  The  reaolta  of  thia  firat  phaae  are  then  pooled, 
coded,  and  averaged.  For  the  aecond  round,  Um  panel 
reoeivea  the  pooled  teaaha  of  round  one,  and  thoae 
whoee  initial  leaponaea  differ  naarkedly  from  the  group 
norm  are  aaked  either  to  modify  their  original  responae 
if  they  wiah  to  do  ao,  or  to  provide  a  defenae  of  their 
poaition  by  atating  their  reaaona  more  fully.  Theae 
reaponaea  are  again  pooled  and  circulated  once  more. 
The  cycle  can  be  repeated  aa  long  aa  the  proceaa  yielda 
aignificant  changea  in  the  expreaaed  opiniona. 

The  Ddphi  technique  ia  an  outgrowth  of  the  long¬ 
term  intereat  of  Hdmer  and  Readier  in  the  development 
of  projective  techniquea  appropriate  to  the  inexact 
aciencea.  Ita  diief  advantage  ia  ita  avoidance  of  time- 
oonanming  committee  meetinga  which  are  the  conven¬ 
tional  aubatitute  for  thia  kind  of  intellectual  inter¬ 
change.  The  avoidance  of  meetinga  afforda  a  further 
advantage  in  that  it  minimiaea  the  effect  of  paychologi- 
cai  preacurei  commonly  induced  by  a  committee  setting, 
aoch  aa  the  familiar  **band  wagon"  effect  (giving  rise  to 
qmrioua  ctmaenaua,  or  what  might  be  called  the  “hail- 
to-the-duef*’  syndrome).  A  further  distortion  that  is 
sometimes  observed  in  the  committee  process  is  the 
rigid  defense  of  pnblidy -expressed  views  because  their 
Mbsequent  modification  is  thought  to  imjdy  lack  of 
conviction  or  egregious  error.  In  addition,  the  repeated 
circulation  of  pooled  responses  to  all  panel  members  is, 
in  itself,  a  povrerful  educatioruJ  drvke  which  serves  to 
creste  a  common  fund  of  knowledge  and  opinion  on 
srhich  to  base  one’s  forecasts.  The  committee  setting 
affords  no  opportunity  for  the  letsureiy  contemplation 
of  divergent  views,  so  that  ita  educational  value  is 
aevecsiy  limilad. 

The  aath<ws  recognise  certain  difftcukies  with  this 
tnehnique,  to  whidi  may  be  added  one  or  two  criticisins 
of  onr  osm.  Ihey  admit  to  the  dtficuhy  of  decidfaig. 
initially,  who  b  aa  expert.  Ptraona  who  enjoy  the 
highest  pwfesitnnel  r^eaations  in  a  given  aaey 
be  fnlly  b  tnne  srith  the  presam.  but  they  are  not 
Moaaaarttr  harhingen  of  the  fnture.  Partknlarly  when 
seeking  long-range  forecast,  it  may  he  deairehle  to 


obtain  a  mix  of  older  and  younger  minds,  so  that 
exclusive  reliance  on  the  criterion  of  established  repu¬ 
tation  may  be  self-defeating.  Further  difficulties  stem 
from  the  inevitable  delays  in  obtaining  responses  from 
laggard  panel  members,  and  the  attrition  of  the  panel 
itself. 

A  more  serious  problem  is  the  design  of  the  question¬ 
naires  themselves.  In  the  Gordon-Helmer  study,  the 
questionnaires  consisted  mostly  of  lists  of  statements 
expreming  possible  future  developments  in  particular 
subject  areas.  The  panel  members  were  aaked  to  check 
one  of  aeveral  boxes  expressing  their  opinion  as  to  the 
effectiveness  of  the  development  in  question  in  achiev¬ 
ing  a  qiecified  goal  (the  categories  offered  ranged  from 
nil  to  high).  They  were  then  asked  to  express  their 
opinion  as  to  the  probability  that  the  specified  develop¬ 
ment  would  occur  by  checking  “never”,  “maybe”,  or 
“ceitain”.  Where  appropriate,  similar  categories  were 
offered  relating  to  the  expectc'i  time  that  the  de’'elop- 
ment  in  question  was  deemed  likely  to  be  achieved,  its 
overall  desirability,  and  the  like.*®  Although  the  com¬ 
position  of  these  items  reflects  the  careful  work  of 
CTpeits  in  the  fields  of  concern,  the  desire  to  minimize 
the  use  of  the  panel  experts’  time,  and  the  interest  in 
facilitating  the  construction  of  averages  and  similar 
summaries  were  met  at  the  expense  of  open-ended 
questions  which  might  have  permitted  qualified  re¬ 
sponses.  One  problem  with  exclusive  reliance  upon 
box-checking  or  multiple-choice  questionnaire  tech¬ 
niques  is  that  they  invite  rapid,  off-the-cuff  lesponses 
wh’ch  may  lack  the  careful  reflection  which  is  actually 
called  for.  A  further  problem  which  the  authors 
recognize  is  that  both  the  questionnaire  items  and  the 
response  alternatives  provided  are  too  brief  and  cate 
gorical  to  permit  serious  qualification.  Hence  there 
exists  considerable  ambiguity  and  imprecision  in  both 
the  stimuli  and  the  responses  they  elicit. 

rinally,  it  should  be  recognised  that  the  conventional 
committee  setting  for  which  the  Delphi  technique  is  a 
substitute  offers  a  number  of  advantages  as  well  as 
disadvantages.  The  interplay  of  judgments  and  influ¬ 
ences  that  occur  in  face-to-face  association  can  be  a 
valuable  s-fjunct  in  the  development  of  reasoned  views 
on  a  given  subject.  Only  through  such  contact  can  each 
member  of  the  group  perceive  the  nuancm  of  mmning 
and  intenaitT  of  convktiont  which  are  of  vital  im¬ 
portance  in  assessing  the  true  significance  of  expressed 
ideas  Much  of  thw  contextual  matter  k  lost  when  we 
reaort  to  written  tomiaunkalion.  Nererthdeaa,  in  a 
srorld  wboae  experts  are  always  in  short  supply  and 
subject  to  heavy  demands,  the  Diriphi  tediniqiM  k 

"Csr^oa  sod  Hrlowr,  fhU.,  tpniadii. 
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certainly  a  worthwhile  approach  to  consider  in  eliciting 
some  systematic  expression  of  considered  views  on  the 
future  prospects  in  any  area  of  interest.** 

In  concluding  this  discussion  of  techniques  of  pro¬ 
jection,  it  may  he  helpful  to  return  to  a  brief  con¬ 
sideration  of  first  principles.  It  is  axiomatic  that  no 
projection  can  he  more  reliable  than  the  data  on  which 
it  is  based.  Much  of  the  criticism  and  controversy 
surrounding  long-range  projections  involves  differences 
whose  magnitude  is  smaller  than  that  of  the  band  of 
error  surrounding  the  underlying  data.  If,  for  example, 
the  estimated  current  population  of  working  ages 
reflects  an  undercount  of,  say,  3  percent,  the  derived 
labor  force,  ceteris  paribus,  will  have  a  similar  error. 
It  follows  t'lat  a  projection  of  the  population  and  labor 
force  will  reflect  that  much  error  as  a  minimum,  so  that 
ahemative  projections  which  differ  by  that  amount 
cannot  be  differentiated  meaningfully.  In  other  words, 
even  our  point  predictions  should  be  regarded  as 
interval  predictions  wherein  a  range  of  values  are 
equally  consistent  with  a  given  set  of  assumptions.*^ 

The  problem  or  errors  in  the  original  observations 
is  far  more  extensive  than  merely  the  undercount  of 
the  population  in  a  given  census.  All  observations  of 
social  and  economic  data  are  afflicted  with  uncertainty 
due  to  errors  of  measurement  and  classification. 
Morgenstem’s  criticism  of  economic  research  activities 

"More  recenllv,  Helmer  (Helmer-HirtcHberg)  hat  linked  hit 
proposed  ute  of  the  Delphi  technique  to  a  far  more  sweeping 
proposal  for  the  establishment  of  an  institute  for  social  plan¬ 
ning  similar  to  that  proposed  (but  in  softer  terms)  br  Bertrand 
de  Jourenel  (op.  cit.,  chapter  20).  See  Olaf  Helmer-Hirtch- 
berg  (89).  The  initial  ideas  for  this  technique  and  its  applica¬ 
tion  were  developed  by  Hciraer  and  Kescher  (88).  The  findings 
of  an  earlier  study  on  the  use  of  eiperts  in  developing  forecasts 
are  informative  in  this  regard.  See  A.  Kaplan  el  al.  (97).  These 
authors  found  ihct  group  discussion  and  the  requi-~‘ment  that 
specific  predictions  be  justified  or  explained  by  the  predictor 
led  to  improved  predictions. 

".A  classic  analysis  of  errors  in  decennial  census  coverage  is 
.An.sley  J.  Coale  (27).  /.  careful  analysis  of  the  eoicrag*-  of  the 
1960  census  is  that  of  Jacob  S.  Siegel  and  Melvin  Zclnik  (IT2). 
The  recommendations  of  Joseph  Steinberg  following  the  Siegel- 
Zelnik  paper,  p  86(f,  are  noteworthy  for  their  implieaiioas  re¬ 
garding  demographic  and  labor  force  projections.  Tor  a  more 
detailed  disettaaion  of  the  policy  implications  of  census  nitdrr- 
CDverage  of  particular  population  gr-vuna,  see  Jacob  Siegel 
i/73). 


in  general  and  economic  forecaats  in  particttlar  is 
fundamentally  a  criticim  of  the  wideqiread  tendescy  to 
ignore  or  undereatimate  these  enora  in  arriving  at 
results  which  are  apurioualy  preciee.  His  conclnding 
recommendations  are  eepecially  incisive:  First,  good 
data  are  needed  if  economic  (or  any  other)  theory  is 
to  enjoy  any  testable  relevance  to  the  real  world. 
Second,  if  the  errors  found  in  the  data  are  attributable . 
to  inherent  difficulties  of  observation  and  measurement, 
they  can  be  reduced  only  by  a  gradual  improvement 
of  the  pertinent  methods;  but  if  they  are  due  to  inten¬ 
tional  falsification,  fauhy  memory,  or  the  inability  to 
obtain  truly  random  samples  for  observation,  the  data 
in  question  must  be  discarded  or  interpreted  in  the 
light  of  a  statistical  theory  (yet  to  be  developed)  which 
could  estimate  and  correct  for  these  distortions.  Third, 
far  greater  effort  must  be  expended  by  data-producing 
agencies  to  estimate  and  publicize  the  errors  associated 
with  the  statistics  they  issue.  Finally,  economists  (and, 
by  implication,  other  social  scientists)  must  devise 
more  flexible  techniques  for  handling  data  with  a  wider 
range  of  accuracy,  rather  than  proceeding  on  the 
assumption  that  their  data  are  sufficiently  accurate  to 
warrant  the  analytical  techniques  that  happen  to  be 
currently  in  favor.** 

For  the  practical  forecaster,  Morgenstem’s  critique 
contains  little  that  is  new  or  particularly  shocking.  As 
noted  previously,  the  forecaster  might  usefully  apply 
the  estimated  band  of  error  surrounding  his  basic  data 
to  the  forecast  values  themselves,  thus  obtaining  t  kind 
of  minimal  interval  estimate  instead  of  a  point  pre¬ 
diction.  He  might  also  draw  certain  comfort  from  the 
realization  that  the  researcher  cannot  escape  uncer¬ 
tainty  merely  by  confining  himself  to  the  past  and 
present. 

In  the  next  chspter,  we  turn  to  problems  related 
specifically  to  long-range  projections. 

"Oskir  Morgriiilern  (735),  p.  3Q2B.  For  i  brief  tuamwry  of 
thiv  work  ind  its  wvrre  ret-ominenda'ioiu.  tee  MorgeaMcra 
1 736).  Morgrnviern’v  criticism  of  dilt-produeiog  ageecin,  par- 
liciitarly  ihosr  of  the  Fnlcral  (^vcmmcol,  for  their  failure  to 
concern  themselves  with  errors  in  the  dila  they  pablisii  awt 
wi-h  a  sharp  rrbullal  by  Raymond  T.  Bowman  (f6).  Bosnaaa’a 
most  telling  point  is  liis  long  but  partial  listing  of  govenuaenl 
puMieaiions  relating  spreifically  to  errors  in  ibe  data  prodiicod 
by  the  ponieular  agencir*. 
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li.  MAJOR  PROBLEMS  IN  LONG  RANGE  PROJECTION 


Hw  dm'dopoieiit  of  tong>rfnge  projections  gives  rise 
to  a  vifiety  of  probfenns,  some  of  which  are  primarily 
tedmical  while  others  involve  broad  philosophical 
ianen  and  qaesdons  of  poBcy.  Furthermore,  these 
problenu  are  oommmiiy  irterrdated,  so  that  solutions 
sdiich  are  lecbnkaliy  feasible  may  be  problematic  for 
reasons  of  pt^cy,  or  on  philosophical  grounds. 

In  considering,  first,  tlw  technical  problems,  it  may 
be  he^ful  to  dktinguiah  four  major  prcblem-creas: 
(1)  The  statement  of  the  assumptions;  (2)  the  selec¬ 
tion  and  evaluation  of  the  data;  (3)  the  treatment  of 
the  data;  and  (4)  tlm  ditermination  of  the  number, 
range,  and  time  span  of  the  projections  themselves. 

A.  SMrmmt  of  the  auumptions 

*n>e  roost  common  plea  of  those  who  seek  to  under¬ 
stand  .lOW  a  projection  was  devdoped  in  order  to  ust 
it  intelligently  is  that  the  underlying  assumptions 
should  be  ma<le  more  explicit.  Since  technicians  are 
usually  greatly  concerned  to  do  likewise,  it  is  surprising 
that  they  seldom  men  the  demands  of  their  critics  in 
this  respect.  One  difiknilty  may  rest  in  the  meaning  of 
the  term  “explicit.”  Ita  ambiguity  can  be  illustrated  bv 
means  of  the  conventional  atatement  of  assumptions  to 
be  found  in  many  publications  of  long-range  projo  ■ 
tioiM  of  the  labor  force**  The  statement  typically 
aaaerts  the  assumption  that  “there  will  be  no  major 
war,  prolonged  depression,  or  other  profound  disturb- 
ame  during  the  period  covered  by  the  projection.” 
Such  a  statement  is  formally  explicit  in  the  usual  sense 
of  the  term,  but  when  the  user  or  critic  demands  ex¬ 
plicitness,  he  is  asking  for  more  than  the  use  uf  clear 
language  and  an  open  statement  of  the  ceteris  paribts 
aiMimptions.  Ideally,  he  wants  to  know  the  precise  effe<'t 
of  the  adoption  of  the  assumption  in  question  on  the 
reouhsnt  projection.  In  other  words,  to  use  Baumol’s 
distinction  once  more,  he  watits  the  assumptions  to  be 
transparent  as  well  as  explicit.  This  ideal  can  only  le 
met  when  an  aasumption  can  be  expressed  as  a  quanti¬ 
tative  dement  in  an  equation,  to  that  the  r^ect  <>( 
r^aining,  removing,  or  altering  the  given  assumptirn 
can  be  readily  m»aared.  Short  of  (his  ideal,  the  ex- 

"Srr,  for  eiAMipic,  5opku  C^prf  Sfutrf  CsrfinlJr 
p.  3%  •ad  tH«  PUttfUas  Aaiocitiion  p.  2,  Cnja- 

•  CBiHlif  rr<Mr)i  cm  tllH  typr  of  frikrysl  ratrat  «o 
hf  lam'aiRrra  ta  ttaiidT  *^afi  aroaoentr  (orrca«t  madr  aad^^r 
ilbr  candilMA  iWt  ikUHtt  a  daprvaaicM  nor  an  ieftatKM  «K«!1 
ttcmr  of  tilll#  pmdbrftw  puwi’r.^  T-fnilr  Cmnhrrs  ^T6i 
Mm  rre^fy.  tka  Bmaa  ol  Km  allrtaplrd  *o 

pmrida  BMT  duaiWd  atalftnNt  of  tKa  ataoMad  rofidiltou 
vndarlTinc  mw  af 'm  Ua  aiaaipW,  Hooatd  V 

SiamKWf  idl). 


presaion  of  assumptions  can  only  be  improved  by  the 
exercise  of  greater  care  in  their  formulation.  One  can 
argue  that  the  kind  of  statement  referred  to  above  is 
essentisily  s  wsy  of  saying  that  a  long-range  projection 
is  usually  prepared  on  the  assumption  of  underlying 
Etructursl  stability.  If  so,  it  could  only  be  made  more 
explicit  by  means  of  a  detailed  description  of  the 
6]>ecific  structursl  elements  whose  continued  stability 
is  assumed,  together  with  some  estimate  of  the  probable 
impact  upon  the  projection  as  a  whole  of  a  failure  of 
any  of  these  elements  to  remain  stable  over  the 
projection  period. 

When  we  consider  long-range  projections  of  labor 
force  in  this  context,  the  demand  for  greater  explicit¬ 
ness  of  assumptions  is.  in  effect,  s  demand  for  a  re¬ 
search  effo.t  of  u..,, .  ecedented  scope.  Consider,  for  ex¬ 
ample.  the  projection  of  labor  force  participation  rates 
for  young  adult  men  and  women  age  16  to  19  years 
to,  let  us  say,  1985.  In  developing  such  a  projection, 
conventional  BLS  practice  has  been  to  examine  past 
trends  in  tlie  worker  rates  of  the  major  subgroups  of 
this  population.  These  subgroups  are  themselves  de¬ 
fined,  within  the  limits  of  available  data,  in  terms  of 
major  social,  demographic,  and  economic  characteris¬ 
tics  which  have  been  associated  in  past  experience  with 
important  differences  in  the  propensity  to  enter  the 
labor  force.  The  more  obvious  of  these  charact 
istics  are  age,  sex,  school  enrollment,  and,  for  women, 
marital  status  and  the  presence  of  young  children. 
.According  to  the  usual  BLS  procedure,  the  projected 
labor  force  for  this  age  group  in  1985  would  be  pre¬ 
pared  hv  the  following  procedure: 

Step  1.  Since,  in  1%7,  not  all  of  the  individuals  who 
will  be  16  to  19  years  old  in  1985  have  yet  been  born,  it 
is  necessary  to  review  existing  population  projections 
for  this  group  In  (he  light  of  available  evidence  con¬ 
cerning  current  trtnds  in  fertility,  in  order  to  select 
the  most  plausible  ,Kipulatiu.  series  to  lie  used  as  a 
base  in  the  projection. 

Step  2,  The  group  as  a  whole  is  then  classified  i-ito 
the  several  subgroups  defined  by  the  major  differen¬ 
tiating  rharat  tr»-.slics  noted  above.  This  categorixation 
is  limited,  of  course,  hv  the  available  data.  It  is  im¬ 
portant  ;-v  observe,  at  this  point,  that  the  pertinent 
characteristics  of  this  age  group  t  its  school  enrollment, 
marital  status,  etc.  (  relate  to  10R5,  not  to  the  present. 
This  implies  that  separate  projections  must  be  made 
of  each  of  these  ^  haraeteristics  to  the  target  date,  or 
existing  projrrt-  ms  must  be  utilixed.  Csmmon  BI.S 
practice  has  been  to  utilixe  the  available  projeeiiotvs  of 
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the  Bureau  of  the  Census  in  this  work  so  far  as  possible. 

Step  3.  Past  and  current  trends  in  the  rates  of  labor 
force  participation  of  each  subgroup  arc  then  obtained 
and  extrapolated  to  the  target  date. 

Step  4.  Past  and  current  trends  in  the  distribution 
of  the  total  group  among  the  several  categories,  as 
projected  to  the  target  date  in  Step  2,  are  then  combined 
with  the  projected  worker  rates  obtained  in  Step  3  to 
obtain  a  projected  labor  force  for  each  of  the  com¬ 
ponents  of  the  total  group. 

Step  5.  The  projected  labor  force  of  each  subgroup 
is  then  combined  to  yield  a  projected  labor  force  for 
the  total  age-group  as  of  the  target  date. 

When  the  demand  for  a  more  explicit  statement  of 
assumptions  is  considered  in  the  light  of  the  above 
account  of  conventional  procedures,  it  is  clear  that  the 
pursuit  of  greater  explicitness  can  only  be  terminated 
arbitrarily,  or  by  the  limits  of  available  time,  patience, 
and  resources.  In  the  above  illustration,  one  might  state 
that  it  is  assumed  that  men  and  women  in  this  age 
group  will  continue  to  pursue  formal  education 
through  the  equivalent  of  4  years  of  high  school  for  the 
overwhelming  majority  and  on  into  college  for  a 
growing  minority.  Further,  it  is  assumed  that  a  sig¬ 
nificant  minority  of  young  women  in  the  age  group  will 
continue  to  marry  and  begin  childbearing.  It  might  well 
be  added  that  the  projection  assumes  no  radical  change 
in  the  structure  of  financial  assistance  offered  in  sup¬ 
port  of  educational  activity,  so  that  substantial  propor¬ 
tions  of  students  in  this  group  will  continue  to  seek  and 
find  paid  employment  while  going  to  school.  It  is  also 
assumed  that  there  will  be  no  radical  change  in  either 
the  desire  for  marriage,  the  desire  for  children,  or  the 
ability  to  avoid  childbearing  among  these  young 
people.  This  implies  an  assumption  of  stability  in  values 
and  motivations  regarding  marriage  and  family  growth. 
Further,  it  is  assumed  that  prevailing  conditions  of 
work,  study,  leisure,  prevailing  definitions  of  these 
activities,  and  prevailing  opportunities  for  choosing 
among  them  will  remain  substantially  unchanged. 
Finally,  it  is  assumed  that  the  prevailing  definitions  of 
the  labor  force  as  it  relates  to  this  age  group  will 
remain  in  force. 

At  this  point,  one  could  take  up  each  of  these  as¬ 
sumptions  so  as  to  reveal  the  further  assumptions 
which  they  entail,  in  an  effort  to  justify  their  adoption 
in  preference  to  possible  alternatives,  and  so  on  ad 
nauseum.  The  point  to  be  understood  here  is  that  one 
must  consider  the  very  real  limitations  of  time,  re¬ 
sources,  and  publication  space  available  to  those  who 
prepare  projections,  and  the  equally  real  limit  to  the 
amount  of  detail  of  this  kind  which  can  usefully  be 


absorbed  by  the  users  of  a  projection.  One  virtue  of 
this  kind  of  detailed  recital  is  that  it  alerts  the  user  of 
the  projection  to  the  network  of  assumptions  which  is 
all  too  easily  overlooked  when  he  quickly  passes  over 
the  glib  assertion  that  the  given  projection  assumes  that 
no  radical  structural  change  will  take  place.  However, 
no  matter  how  detailed  a  verbal  account  of  assumptions 
may  be,  it  cannot  provide  the  user  with  the  kind  of 
information  he  needs  in  order  to  assess  the  real  sig¬ 
nificance  of  the  projection.  For  this  purpose,  he  would 
need  to  know,  in  quantitative  terms,  the  effect  of 
different  parameters  relating  to  each  of  the  assumed 
characteristics  upon  the  resultant  projection,  together 
with  the  probabilities  relating  to  the  likelihood  of 
continued  stability  of  the  factors  to  which  each  of  the 
assumptiens  relates.^® 

The  above  considerations  bring  us  to  the  second 
point  in  regard  to  the  demand  for  more  explicit 
assumptions — namely,  this  is  in  effect  a  demand  for 
more  quantitative  procedures.  As  such,  it  can  only  be 
met  to  the  extent  that  factors  which  are  believed  to  be 
relevant  to  a  given  projection  can  be  expressed  quanti¬ 
tatively.  In  seeking  to  accomplish  such  quantification, 
the  technician  runs  the  very  real  risk  of  imposing  an 
artificial  structure  upon  the  data.  If,  on  the  other  hand, 
he  eschews  consideration  of  non-quantifiable  factors  in 
order  to  avoid  the  charge  of  subjective  bias  or  im- 
prec’iion,  he  runs  the  still  greater  risk  of  ignoring 
sign'ficant  factors  which  affect  the  phenomenon  he 
seeks  lo  ."'roject.  The  heroic  solution  to  this  dilemma 
is  of  course  familiar  to  every  graduate  student  in  the 
behavioral  sciences;  if  a  factor  of  alleged  importance 
is  in  fact  significant,  its  impact  must  be  measurable. 
Hence  only  those  factors  whose  impact  is  measurable 
should  be  regarded  as  significant.  Unfortunately,  where 
the  available  data  are  of  limited  scope  and  quality  ,  it 

"A  further  complication,  particularly  in  the  case  of  long- 
range  projections,  is  tiie  reversahility  of  causal  sequences  among 
associated  factors.  For  example,  conventional  projections  of 
labor  force  supply  assume  a  given  population  projection  aa  a 
starting  point.  However,  fertility,  mortality,  and,  a  fortiori, 
migration  are  all  affected  Ijy  the  economic  siturtion,  which 
therefore  influences  the  labor  force  directly,  via  job  opportuni¬ 
ties,  and  indirectly,  via  longrun  deniographic  changes.  We  are 
thus  confronted  by  a  causal  network  rather  than  a  causal  chain. 
The  two-way  relationship  between  population  growth  and  eco¬ 
nomic  development  has  of  course  been  recognized  in  a  large 
number  of  studies.  See,  for  example,  Irma  Adelman  (2),  Robert 
M.  Dinkel  (45),  and  Stephen  Enke  (50).  An  excellent  sum¬ 
mary  of  the  development  of  theory  relating  to  demographic- 
economic  interrelations  is  A.  L  Levine  (f08).  For  a  recent 
analysis  reflecting  some  of  these  interrelations,  see  Peter  E. 
Haase  (79).  An  early  exposition  of  population  change  as  a 
factor  in  labor  force  growth  is  the  classic  study  by  Seymour  L. 
Wolfbein  end  A.  J.  Jofie  {204). 
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w  iwoamrjr  to  rdy  upon  i  priori  theory,  experience, 
and  even  intuition  in  order  to  determine  the  tignifi* 
ranM  of  facton  vrhose  operation  is  hidden  from  view. 
It  is  sometimes  posable  to  incorporate  nonnumerical 
dements  in  the  form  of  “donuny”  variables  and  thns 
indode,  in  some  sense,  factors  which  would  otherwise 
hate  been  excluded  because  they  could  not  be  expressed 
qiumtitatlvdy,  but  this  parctice  is  rot  always  adequate 
or  feasible." 

A  third  point  concerning  the  demand  for  more 
eq>licit  assumptions  relates  to  the  criteria  whereby 
clarity  is  to  be  determined.  Here,  we  encounter  the 
dilemma  discussed  previously — t  carefully  worded 
statement  of  aasumptiona  such  as  those  described  in 
regard  to  a  projection  of  the  labor  force  of  16-  to  19- 
year-olds  may  be  perfectly  dear  and  explicit  to  the 
practical  user,  while  remaining  full  of  ambiguity  to  the 
iecimic>a.n.  Conve.'sely,  a  stale  nent  which  is  adequately 
explicit  to  the  technician  will  commonly  mystify  anyone 
lacking  be  requisite  training  in  quantitative  methods 
and  terminology  In  short,  it  is  necessary  to  recognize 
that  projections  are  used  and  must  be  undr-stood  by 
nou-technicians  whose  demands  for  explicit  statements 
of  assumptions  cannot  be  satisfied  by  a  list  of  equations. 
B.  Sdecticn  and  Evaluation  of  Data 

As  noted  previously,  relunce  upon  simple  extrapola¬ 
tive  techniques  in  developing  projections  oilers  one 
important  advanta"* — suen  a  projection  tan  be  ob¬ 
tained  on  the  ba^  past  trends  in  the  phenomenon 
being  projected,  wicnout  resorting  to  an  anal/sif'  of  its 
underlying  medienions  or  cause!  prc'’e8f.as.  Hov;ever, 
the  ultimate  interests  of  neither  the  us^r  nor  the  tech¬ 
nician  can  be  mH  in  this  mamier,  since  it  does  not 
provide  an  understanding  of  the  factors  which  de¬ 
termine  the  temporal  movement  of  the  phenomenon  in 
quertion.  Thus  attention  is  turned  to  the  discovery  of 
factors  associated  with  the  phenonemon  of  primary 
interest,  and  to  the  measurement  of  their  impact.  At 
thb  point,  the  need  for  theory  to  guide  the  analyst  in 
the  selection  of  relevant  variables  is  paramount.  In  the 
absence  of  adequate  theory,  and  given  the  availability 
of  automatic  data  processing  equipment  of  growing 
versatility,  the  analyst  can  nov'  feed  all  his  data  into 
the  hopper  and  passively  await  the  .omputer  printouts 
which  will  tell  him  which  variables  are  significantly 
associated  with  his  dqiendent  variable.  The  only  dif¬ 
ficulty  with  this  approach  (apart  fioni  the  very  real 
possibility  that  the  analyst  can  be  overwhelmed  by  the 
computer  output)  is  that  the  occurreiice  of  associations 

"This  is  not  always  a  hapjiy  solution,  since  most  qualitafve 
factors,  by  their  very  nature,  are  not  readily  dsssiiiSble  as 
merely  present  or  absent  in  a  given  instance. 


cannot  be  onderstood  or  explained  without  resort  to 
theory.  Furthermore,  anUyats  who  seek  to  derive  con¬ 
clusions  on  the  basis  of  associations  discoversd  in  this 
manner  tend  to  engage  in  the  dubious  practice  which 
Professor  Mosteller  used  to  deride  as  “capitalizing  on 
chauce.”  That  is,  the^.  sedc  to  attribute  significance  to 
meaningless  associations  that  are  bound  to  appear 
eventually,  given  a  sufficient  mass  of  input,  particularly 
when  the  observatiods  relating  to  the  several  variables 
are  not  entirely  independent  of  one  another,  as  is  com¬ 
mon  in  economics. 

Unfortunately,  theory  rdating  to  the  factors  in¬ 
fluencing  the  entry  or  withdrawal  of  persons  from  the 
labor  '  vee  is  not  entirely  adequate  for  the  reeds  of 
long-range  forecasting.  Furthermore,  the  kind  of  in¬ 
formation  that  is  available  itself  reflects  the  paucity  of 
relevant  behavioral  theory,  so  that  the  analyst  is 
forced  to  woilc  with  economic  data  primarily,  even  if 
he  suspects,  as  does  the  present  writer,  that  the  major 
influences  on  the  propensity  of  persons  to  enter  or  leave 
the  labor  force  are  social  and  psyclological  as  much  as 
economic  in  nature."  In  short,  the  development  of  a 
more  adequate  basis  for  labor  force  projections  is 
hampered  by  tbe‘  relative  absence  of  relevant  non¬ 
economic  data. 

In  this  situation,  two  courses  of  action  may  be 
recommended:  First,  it  is  necessary  to  work  with  what 
is  available  in  the  knowledge  that  both  our  failures  and 
successes  will  prove  instructive.  Second,  it  is  necessary 
to  develop  a  more  adequate  behavioral  theory  relating 
to  labor  force  activity,  on  the  basis  of  which  we  can 
specify  the  kinds  of  information  needed.  Thii  implies 
a  need  for  far  greater  collaboration  between  eco¬ 
nomists,  who  have  dominated  the  field  largely  by 
default,  and  other  social  scientists.  We  shall  elaborate 
on  this  point  in  the  concluding  chapter. 

C.  Treatment  of  the  Data 

In  making  projections  by  extrapolative  teenniques, 
one  encounters  all  of  the  problems  associated  with  time 
series,  sooner  or  later.  The  ."'st  of  these  is  the  problem 
of  changing  definitions  and  changing  coverage  which 
impair  the  comparability  of  data  through  time.  In 
working  with  population  esiimates  for  the  United  States 

“Deapite  the  limilations  of  available  theory  and  the  even  more 
aerious  limitations  of  available  data,  a  number  of  analysts  have 
made  sisniiicnnt  inroads  toward  explaining  the  factors  affecting 
the  labor  force  ac  ivity  of  particular  aubgroups  of  the  popula¬ 
tion.  See,  for  exit.nple,  Glen  Cain  (23,  24),  and  Jacob  Mincer 
(129,  130).  Profewr  Mincer  is  continuing  his  research  into 
the  relation  between  labor  force  participation  and  unemploy¬ 
ment.  Among  :he  others  who  have  made  important  contribu¬ 
tions  in  this  area  are  Qarence  D.  Long  (113),  Alfred  Telia 
(191, 192) ,  and  W.  Lee  Hansen  (82) . 
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over  the  postwar  period,  for  example,  a  number  of 
elementary  but  easily  overlooked  examples  come  to 
mind.  Do  the  estimates  include  or  exclude  Alaska  and 
Hawaii  throughout  the  period  of  interest?  Are  the 
worker  rates  based  on  the  total  population,  including 
Armed  Forces  abroad,  or  are  they  bas^  on  the  total 
resident  population,  or  the  total  civilian  jiopulation,  or 
the  civilian  noninstitutional  population?  'Similarly,  the 
recent  change  in  the  definition  of  the 'population  of 
working  age  from  14  and  over  to  16  and  over  necessi* 
tates  caution  in  comparing  statistical  aggr^ates 
through  time. 

A  second  problem  relates  to  the  treatment  of  observa¬ 
tions  that  are  out  of  line  or  atypical  in  a  given  time 
series.  In  developing  an  extrapolation,  the  technician  is 
confronted  with  a  decision  here,  and  one  which  re¬ 
quires  the  exercise  of  considerable  judgment.  He  can 
simply  retain  all  observations  as  given /thus  avoiding 
a  judgment  that  may  be  both  complicated  and  hard  to 
defend) .  However,  he  thereby  runs  the  risk  of  seriously 
distorting  his  results.  Alternatively,  he  can  examine  the 
historical  circumstances  surrounding  the  deviant 
observation  to  determine  whether  it  can  be  explained 
by  actual  social  or  economic  conditions  or  must  instead 
be  attributed  to  chance  fluctuation  or  a  peculiar  com¬ 
bination  of  distorting  factors.  Whatever  the  outcome 
of  this  examination,  the  technician  must  then  decide 
whether  the  observation  in  question  should  be  accept^' 
and  given  its  full  weight,  assigned  a  reduced  weight, 
smoothed  Into  line,  or  left  out  of  consideration  entirely. 

A  familiar  example  can  be  seen  in  the  labor  force 
participation  rates  of  men  20  t<i  24  years  old  in  the  im¬ 
mediate  postwar  period  (19J!(?— 52).  These  rates  rose 
very  rapidly  from  84.9  percent  in  1947  to  92.1  percent 
in  1952 — an  increase  of  7  percentage  points  in  5  years. 
In  extrapolating  these  rates  (which  have  declined 
slowly  and  irregularly  since  1953)  the  technician  must 
decide  whether  the  rise  in  the  1947—52  period  should 
be  allowed  to  influence  the  extrapolation  proportion¬ 
ately  or  whether  it  should  be  left  out  of  consideration 
entirely.  If  he  chooses  the  former  course,  he  extrapo¬ 
lates  on  the  basis  of  a  trend  line  through  the  observed 
rates  for  the  1947-66  period,  obtaiiiing  a  constant  or 
slightly  increasing  trend  into  the  future.  If,  on  the  other 
hand,  he  leaves  out  the  1947—52  period,  his  extra¬ 
polation,  derived  from  the  trend  for  the  1953-66 
period,  will  yield  a  gradually  declining  rate  of  labor 
force  participation  for  this  group. 

In  the  above  example,  the  analyst  can  readily  surmise 
that  the  postwar  enrollment  of  veterans  unider  the  GI 
bill  held  down  their  worker  rates,  and  hence  accounted 
for  the  low  1947-48  rates  of  the  2()-to  24-age  fioup  as 


a  whole.  He  would  therefore  be  likely  to  discount  the 
increase  from  1947  to  1952  as  reflecting  on  unusual 
condition  which  he  would  not  want  to  incorporate  as 
one  of  the  assumptions  in  his  projection.  Unfortuiutdy, 
not  all  of  the  quirks  in  a  statistical  time  series  are  so 
readily  explained  or  dismissed,  so  that  the  role  of  eiq>ert 
judgment  is  crucial.  This  implies,  first,  that  the  tech¬ 
nician  must  either  possess  a  thorough  knowledge  of  the 
subject  matter  with  which  he  deals  in  preparing  a 
projection,  or  be  able  to  work  closely  with  subject- 
matter  exjerts.  Second,  the  judgments  which  are  exer¬ 
cised  in  making  projections  must  be  recognised  as  their 
most  essential  component.  Except  for  mechanical  errors 
or  trivial  differences  in  procedure,  any  projection  can 
only  be  criticized  in  terms  of  its  underlying  assump¬ 
tions  and  the  judgments  that  have  been  made  at  critical 
stages  in  its  development. 

A  third  problem  r^arding  the  treatment  of  long 
s41isti':uil  time  series  is  the  relative  weight  to  be  given 
to  earlier  and  more  recent  observations.  Here  again, 
the ’simplest  and  most  easily  defended  procedure,  that 
of  equal  weighting  throughout,  may  not  always  be  the 
most  judicious.  It  is  possible,  for  example,  that  the 
more  recent  observations  are  obtained  through  im¬ 
proved  procedures  or  from  a  better  sample,  and  thus 
provide  a  more  accurate  picture  of  reality  than  the 
earlier  observations.  The  analyst  must  also  consider  the 
possibility  that  one  or  more  periods  within  the  entire 
time  series  may  reflect  underlying  structural  changes 
which  are  inconsistent  with  the  assumptions  of  his 
projection.  An  obvious  example  is  the  projection  of 
labor  force  participation  rates  for  a  population  group 
whose  worker  rates  are  sensitive  to  the  economic  situ- 
-ation.  In  siich  a  case,  a  projection  derived  from  rates 
^^bserved  during  a  recession  period  would  be  inconsist¬ 
ent  with  an  underlying  assumption  of  high  employment 
levels.  In  developing  such  a  projection,  it  would  be 
necessary  to  discount  or  adjust  the  group’s  worker  rates 
during  the  recession  period.  Once  more,  the  crucial  role 
of  judgment  is  evident.^® 

The  above  problems  are  multip’'’ed  when  simple 
extrapolative  techniques  are  abandoned  in  favor  of 
more  complicated  analytical  methods.  In  fact,  a 
fundamental  reason  for  the  hesitation  of  experienced 

"This  problem,  and  the  development  of  techniquts  to  over¬ 
come  it  has  been  a  central  concern  of  Thomas  Deinberg  and 
associates.  See  Thomas  Demberg  and  Kenneth  Strand  (41)  and 
Thomas  Demberg,  Kenneth  Strand  and  Judith  Dukler  (43). 
For  a  brief  summary  of  their  argument,  see  Thomas  Demberg 
and  Kenneth  Strand  (42).  Concern  with  the  possibility  of  hid¬ 
den  unemployment  during  economic  downturns  is  not  recent. 
For  an  earlier  analysis  of  this  phenomenon,  aee  Harold  Wool 
(205). 
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womoMtiidaiu  to  prepare  projecttoiu  beyond  diort- 
range  poioda  (3  to  18  months  ahead)  is  their  knowi¬ 
ng  tfiat  swdi  projections  entail  separate  projections 
of  moat  of  the  variables  in  the  econometric  equation 
qrstem  or  modd.  h  sndi'  a  case,  the  judgments  em¬ 
ployed  by  the  econometrician  are  likdy  to  be  even 
mote  numerous  and  complex  than  ^ose  of  the 
judgmental  forecaster  using  simple  extrapolative 
tedmiqnes.^ 

D.  The  Number,  Range,  and  Timespan  of  the  Pro- 
jeetions 

Decisions  relating  to  the  number  of  ahemative 
projections  to  be  made,  their  range  (i.e.,  the  extent  to 
wfakdi  the  ahemative  series  differ  from  one  another) 
and  their  timespan  (how  long  into  the  future  they 
extend)  can  only  be  reached  upon  a  careful  considera¬ 
tion  of  the  data  and  resources  available  to  the  tech¬ 
nician  and  the  purposes  which  the  projections  are 
intended  to  serve.  In  its  most  general  sense,  a  pro¬ 
jection  is  some  function  of  the  available  data,  so  that  a 
consideration  of  past  trends  and  variations  in  the 
relevant  data  provides  a  good  initial  notion  of  the 
number  and  range  of  projections  which  are  needed  to 
capture  the  likely  variations  in  the  future.  For  example, 
the  trends  and  fluctuations  in  ^e  labor  force  partici¬ 
pation  rates  of  men  and  women  35  to  44  years  old  over 
the  1947-66  period  reveals  a  range  of  0.9  percentage 
points  with  a  high  of  98.2  percent  for  the  men.  For  the 
women,  the  range  is  10.6  percentage  points,  with  a  high 
of  46.9  percent.  Obviously,  projections  relating  to  the 

'omen  in  this  age  group  should  be  greater  in  number 
and/or  far  wider  in  range  than  those  for  the  men  in 
the  corresponding  age  group. 

The  question  of  the  number  of  projections  to  be 
prepared  is  problonatic.  In  preparing  population  pro¬ 
jections,  the  experience  of  the  Bureau  of  the  Census 
has  been  that  two  alternative  sets  (or  series)  of  pro¬ 
jections  tend  inevitably  to  be  labeled  “high”  and  “Lw” 
or  even  “optimistic”  and  “pessimistic”,  with  the 
obvious  pejorative  connotation  of  the  latter  term 
affecting  the  decision  to  adopt  the  series  so  labeled. 
Three  alternative  sets  or  series  are  little  better,  since  the 
users  then  feel  an  overwhelming  pressure  to  adopt  the 

“It  J$  only  fair  to  add  that  experienced  econometricians  are 
unlikely  to  take  long-range  projections  rerionsly  for  the  same 
reasons  that  they  are  reluctant  to  tiy  to  make  them — the  neces¬ 
sary  assumptions  must  be  determined  judgmentally  and  are  diffi¬ 
cult  to  quantify  or  defend.  On  this  point,  see  John  R.  Meyer 
and  Robert  R.  Glauber  il28),  pp.  240-257.  For  a  defense  of 
the  role  of  judgment  by  an  economist  whose  qualifications  as 
scientist  and  eqonometrician  are  above  reproach,  see  Nicholas 
Georgescu-Roegen  (58),  pp.  274f. 


middle  one.  More  recently,  therefore,  census  projections 
of  population  have  appeared  in  four  series,  labeled 
with  the  greatest  possible  neutrality  “A”,  “B”,  “C”,  and 
“D.”***  This  has  afforded  the  several  government  agen¬ 
cies  who  use  the  projections  the  great  satisfaction  of 
(jxpieasing  a  conservative  optimism  by  adopting  aeries 
B,  the  second  highest  of  the  four. 

An  alternative  approach  to  this  question  is  offered  by 
the  National  Planning  Association,  which  has  prepared 
economic  projections  in  three  series — a  target  or  goal 
projection,  a  present  policy  projection  (derived  by  an 
extrapolation  from  past  trends),  and  an  intermediate 
judgmental  projection,  reflecting  the  attainments  which 
are  deemed  the  most  likely  outcomes  of  the  com¬ 
promises  to  be  expected  among  the  inertia  of  custom, 
the  pressures  of  conflicting  goals,  and  aqtirations 
toward  higher  achievement.^^  This  approach  is  far 
more  useful  to  the  user  of  the  projections,  particularly 
in  the  formulation  of  policies,  since  it  affords  a 
plausible  description  of  a  goal  to  aim  for  together  with 
some  notion  of  the  probable  “gap”  between  the  goal 
and  what  might  be  expected  to  ensue  in  the  absence  of 
a  directed  effort.  This  gap  is,  in  a  sense,  a  measure  of 
both  the  need  for  a  particular  program  and  an  estimate 
of  its  maximum  potential  effectiveness.  As  such,  it  can 
be  useful  to  the  policymaker  as  an  expression  of  the 
magnitude  of  a  given  task  and  the  stakes  involved.  ^ 

For  the  technician,  the  instruction  that  he  is  to  pW 
pare  two,  three,  or  four  projections  is  hardly  a  solutii' 
to  his  problems.  He  must  still  select  from  among  a  v^jy 
large  number  of  assumptions  those  fev>  combmations 
which  best  reflect  the  alternative  conditions  which  ,  are 
deemed  worthy  of  separate  projection.  The  natur^  of 
this  task  can  be  readily  appreciated  by  considering 
long-range  population  projections.  The  population  of 
an  area  is  of  course  a  function  of  only  three  variables: 
fertility,  mortality,  and  net  migration.  In  view  of  ob¬ 
served  past  variations  in  these  variables,  one  might 
decide  to  work  with,  say,  four  sets  of  fertility  assump¬ 
tions,  three  sets  of  mortality  assumptions,  and  three 
sets  of  migration  assumptions.  In  their  several  com¬ 
binations,  these  assumptions  would  yield  36  sets  of 
projections,  a  number  which  is  obviously  excessive  for 
any  practical  purposes.  Gf  course,  it  is  likely  that  many 
of  these  combinations  would  resemble  each  other 
closely,  but  the  point  to  recognize  is  that  the  task  of 
the  technician  is  not  limited  .o  spinning  out  the  impli- 

“Jacob  S.  Siegel,  et  al.  (op.  cit.,  1964) . 

“National  Planning  Association,  op.  cit.,  p.  2.  For  an  excel¬ 
lent  account  of  the  rationale  behind  the  NPA’s  alternative  sets 
of  projections,  see  Sidney  Sonenbliini  and  Louis  H.  Stem  (178). 
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cations  of  a  large  number  of  alternative  assumptions. 
He  must  at  some  stage  act  decisively  in  reducing  their 
number  to  manageable  size,  and  in  doing  so,  must 
exercise  judgment  as  to  which  sets  of  assumptions  best 
illustrate  the  plausible  range  of  expectations  regarding 
possible  futures.  In  fact,  he  may  be  expected  to  push 
his  selective  process  to  the  limit,  inasmuch  as  he  will 
often  be  asked  to  identify  the  single  most  likely  pro¬ 
jection — in  other  words,  he  will  be  asked  lo  make  a 
forecast.  It  is  at  this  point  that  his  technical  expertise 
is  of  little  help,  and  his  general  familiarity  with  the 
subject-matter,  viewed  in  its  broadest  social  and  his¬ 
torical  context,  becomes  an  essential  asset.  One  implica¬ 
tion  of  the  foregoing  discussion  stands  out:  No  matter 
how  many  alternative  series  may  be  projected,  the 
practical  needs  of  the  users  will  usually  require  the 
selection  of  a  single  series  for  use  in  the  formulation 
of  policies  and  programs.  That  selection  requires  the 
closest  cooperation  and  mutual  understanding  between 
the  technician  and  the  policymaker. 

Two  considerations  are  paramount  in  determining 
the  timespan  of  a  projection.  The  first  is  the  purpose 
which  the  projection  is  designed  to  serve  and  the  second 
is  the  degree  of  uncertainty  that  is  acceptable.  Here 
.rgain,  some  kind  of  trade-off  must  be  achieved  between 
a  relatively  confident  projection  that  extends  over  too 
short  a  period  to  allow  for  practical  use  and  a  less 
precise  projection  covering  a  longer  timespan.  When 
we  consider  the  rapid  expansion  of  the  band  of  un¬ 
certainty  surrounding  any  projection  as  it  is  carried 
farther  into  the  future,  it  is  indeed  questionable  that 
any  long-range  projection  is  worth  developing.  How¬ 
ever,  their  continued  development  is  assured  by  the 
growing  demand  for  long-range  planning  in  all  sectors 
of  our  social  existence.^^  Furthermore,  such  plans  are 
typically  developed  with  sufficient  flexibility  and  gen¬ 
erality  that  a  considerable  range  of  error  in  the 
projected  magnitudes  can  be  tolerated. 

In  the  manpower  field,  the  timespan  of  projections 
is  dictated  by  the  leadtime  required  to  plan  and  estab¬ 
lish  the  educational  and  training  facilities  needed  to 
meet  tomorrow’s  requirements  for  highly  qualified 

"On  the  need  to  recognize  this  demand  as  a  necessary  and 
legitimate  function  of  government,  see  Jan  Tinbergen  (197),  pp. 
74f.  See  alsd  the  remarks  of  Henri  Theil  (op.  ci(., 'I960), 
Alfred  Sauvy  (158, 159),  and  Richard  A.  Lester  (107). 

The  advamages  of  medium-range  manpower  projections  (ex¬ 
tending  about  5  rears  into  the  future)  are  well  described  by 
R.  Ciossley  (32).  For  »'mployment  forecasts  in  particular, 
Croaaley  argues  that  the  5-year  forecasting  period  affords  suffi- 
'cient  time  for  effective  implementation  of  specific  policies  to¬ 
gether  with  sufficient  precision  for  meaningful  manpower 

planning. 


personnel.  In  the  anticipating  future  needs  for  li4^ 
trained  manpower,  such  as  scientists,  engineers,  and 
physicians,  “tomorrow”  means  10  to  15  years  in  die 
future.  Even  greater  leadtime  is  required  in  planning 
for  financial  and  institutional  support  of  such  programs 
as  the  Social  Security  system  and  other  private  and 
public  pension  and  retirement  systems. 

The  broader  philosophical  issues  which  arise  in 
regard  to  long-range  projections  rdate  to  their  pur¬ 
poses,  their  uses,  and  their  social  consequences.  Since 
these  considerations  are  interrelated  with  questions 
of  public  policy,  they  must  be  viewed  in  the  context  of 
social  and  political  action  designed  to  maintain  the 
society  and  guide  its  progress  in  a  changing  world. 
Perhaps  the  most  fundamental  issue  is  that  of  the  basic 
model  to  be  used  as  a  guide  in  developing  long-range 
projections.  The  central  argument  of  this  essay  is  that 
long-range  projections  cannot  adequately  fulfill  their 
most  important  functions  if  they  are  prq>ared  solely  in 
accordance  with  the  dictates  of  scientific  method,  with 
the  ultimate  goal  of  achieving  predictive  significance. 
This  is  not  an  easy  argument  to  defend,  particularly 
in  an  age  which  equates  truth  and  rationality  with 
science.  However-  this  argument  is  not  an  exhortation 
to  abandon  scientific  procedures.  It  is  rather  a  claim 
that  the  act  of  projection  must  have  an  extra-scientific 
aim.  The  chief  value  of  a  projection  is  its  delineation 
of  possible  futures,  not  its  conditional  prediction  of 
more-or-Iess  likely  outcomes.  Projections  should  serve 
to  open  up  possibilities  and  explore  their  consequences, 
not  reduce  them  to  a  single,  seemingly  inevitable  path. 
The  use  of  scientific  methods  is  indispensable  in  assess¬ 
ing  the  range  of  possibilities  and  determining  tbe 
probable  consequences  of  alternative  courses  of  action. 
But  insofar  as  science  seeks  to  discover  either  deter¬ 
ministic  or  probabilistic  laws  on  which  predictions  can 
be  based,  its  ultimate  purpose  differs  radically,  from 
that  of  projection.  This  difference  and  its  significance 
have  been  aptly  summarized  by  Bertrand  de  Jouvenel, 
when  he  points  out  that  the  ability  to  predict  the  future 
implies  its  inevitability,  which  in  turn  remov^  the 
need  for  projection.  For  this  reason,  he  argues,  our 
interest  rests  with  uncertain  phenomena,  since  only  in 
the  realm  of  the  uncertain  can  we  act  to  influence  our 
destiny.^® 

The  advocates  of  greater  scientific  rigor  in  the  de¬ 
velopment  of  projections  encounter  a  further  difficulty 
when  they  consider  the  impact  of  public  projections 
upon  that  public.  That  is,  they  encounter  the  dilemma 
with  which  social  scientists  are  familiar — the  public 
(i.e.,  social)  expression  of  the  predicted  behavior  of 

"Bertrand  de  Jouvenel,  op.  cit.,  p.  52. 
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JillJbMihfli  of  •  todety  beooniM  an  added  clement  in 
^H^ifili^nalloii  tti  liiiit  bdurior,  which  thoi  cauaea 
'iBi^dhnfthi  dia  behavior  from  what  it  would  odwr* 
^Mdi,  fri  ehort,  we  are  oonfronteu  with  the 
pnUBaa'cd  the  aelf>falfilling  and  self*denying  pro* 
ffcwriae.  In  eomMaring  this  problem,  it  is  important  to 
hMr  in  mind  diat  it  is  only  problemade  for  those  who 
Mi  identify  pro}ections  with  scientific  procedures  and 
godi  dwt  diey  iq[>iiold  the  scientific  aim  of  successful 
fiiojecdmi.  OMonaly,  a  projection  which  is  either  self- 
fuNBing  or  sdf*dea]ring  cannot  meet  this  criterion. 
However,  when  projections  are  viewed  as  statemoits  of 
social  goals  or  as  descriptions  of  possible  futures  which 
may  be  regarded  as  harmful  or  threatening,  their  self- 
frdfiOing  or  self-denying  character  becomes  potentially 
useful.  If  a  projection  swves  to  raise  aspiration  levels 
so  that  the  attainment  of  desired  goals  is  realised,  or 
if  it  provides  advance  warning  of  impending  difficulties 
so  that  plans  are  set  in  motion  to  avoid  these  difficul- 
des,  it  has  fulfilled  its  most  in^rtant  function — one 
that  hears  litde  rdation  to  the  predictive  aims  of 
aeienoe. 

IhOae  who  eqpouae  a  rigidly  scientific  ''onoeption  of 
projeodons  are  led  into  a  consideradon  of  the  above 
{Krobfrm  in  terms  of  highly  convoluted  speculations  re* 
'guding  a  potendally  endless  chain  of  stimulus  and 
response,  in  an  effort  to  incorporate  the  probable  re* 
spmise  to  a  public  projecdon  into  the  projecdon  before 
it  is  made  public.  One  consequence  of  such  speculations 
is  the  asaerdon  that  long-range  projecdoru  of  man¬ 
power  should  be  discouraged  or  at  least  regarded  with 
great  suqricion,  since  dn  r  consequences  cannot  be 
foretold,  and  may  be  harmful.^*  Other  partisans  of 
sdendfic  rigor  have  arrived  at  the  optimistic  conclusion 
that  it  is  possible,  at  least  in  principle,  for  the  tech¬ 
nician  to  incorporate  in  his  projecdon  the  probable 
re^wnae  of  the  public  to  his  announcement  of  the 
projecdon,  thus  avoiding  the  dilonma  of  self-fulfilling 
or  adf-denying  statements.*** 

On  the  basis  of  our  argument  as  to  the  proper  tadt 
of  projecdon,  the  above  view  is  subject  to  two  crid- 
ciama.  First,  predkdve  accuracy  is  neither  a  basic  goal 

*W.  Lee  Hameo  (83) ,  pp.  13ff.  Huuen’e  conmi«ii.ti  are  fol¬ 
lowed  bjr  an  excellent  rejoinder  by  Sol  Swerdloff,  pp.  2Sff.  It  it 
an  nsfortnnate  tmiaai  that  all  knowledfe  nuy  be  mitcaed,  in- 
elading  the  uncertain  knowledce  which  tokee  the  form  of  pio- 
Jeeiloaa.  On  the  other  hand,  the  conceelment  of  knowledse  it 
almoet  certain  to  hate  hamful  conteqaeneet. 

"Emile  Gmnberg  and  Franco  Modidian’  (77).  Compare  the 
apt  remarka  of  Georfe  C  Smith  (177).  In  tociology,  the  prob¬ 
lem  of  the  telf-fnlfilling  prophecy  and  of  reflexite  predictkiit 
aenetally  haa  Ion(  been  reooiniied.  See,  for  example,  Robert  K. 
Merton  (127). 


nor  the  sole  criterion  of  the  worth  of  a  long-range 
projection.  Hence,  the  attempt  to  incorporate  into  a 
projection  a  diain  of  anticipated  reocdons  to  it  implies 
the  sacrifice  of  a  major  objecdve  for  the  achievanoit 
of  a  minor  one.*^  A  second  and  more  serious  objection 
is  that  any  public  projecdon  loses  its  effeedveness 
either  as  forecast,  target,  or  warning  insofar  as  it  comes 
to  be  regarded  as  an  effort  at  manipulation  or  pro¬ 
pagandizing.  The  best  way  to  avoid  the  suspicion  of 
manipulative  intent  is  to  avoid  all  attempts  to  manip¬ 
ulate.  This  means  that  target  or  goal  projections  must 
be  clearly  announced  as  such.  Ihere  purpose  is  to  in¬ 
form  the  public  of  the  possibilities  inherent  in  the 
society.  A  projection  is  manipulative  only  if  it  is 
designed  to  create  an  artificial  atmosphere  of  progress 
and  prosperity,  or  to  exhort  the  public  to  adopt  certain 
goals  by  concealing  or  downgrading  other  alternatives. 
The  attempt  by  any  government  a^cy  to  alter  its 
projections  by  incorporating  the  anticipated  reactions 
of  the  public  to  its  announcement  might,  in  theory, 
improve  its  predictive  accuracy,  but  it  would  hardly 
contribute  toward  easing  the  credibility  gap. 

Even  if  we  grant  that  the  procedures  followed  in 
conventional  sciendfic  invesdgations  do  not  afford  ah 
adequate  pat.idigm  for  the  construction  of  projections, 
the  fullest  possible  use  of  sciendfic  methods  remains 
the  best  protection  we  have  against  the  tendency  to 
reduce  all  projections  to  exercises  in  politically-in¬ 
spired  wish  fulfillment.  Indeed,  many  of  the  functions 
of  projections  call  for  no  more  and  no  less  than  the 
utilization  of  appropriate  scientific  methods.**  For 
example,  projections  may  be  developed  as  a  quasi- 
experimental  substitute  for  the  kind  of  hypothesis  test¬ 
ing  which  occurs  in  a  laboratory  setting,  llie  para¬ 
meters  of  the  independent  variables  of  a  given  equation 
may  be  set  in  accordance  with  a  given  hypothesis  and 
then  changed  to  reflect  a  number  of  alternadve  hypo- 

"Roger  C.  Buck  (22).  In  their  reply  to  this  article,  Grun- 
berg  and  Modigliani  noint  oat  that  their  theorem  expressing  the 
possibility  of  incorp^,.  ing  tne  anticipated  public  reaction  to  a 
prediction  in  the  prediction  itself  does  not  imply  an  endless  re¬ 
gression  of  action  and  reaction,  nor  does  it  require  that  such  a 
regression  be  convergent.  All  that  is  required,  they  assert,  is 
that  the  public  reaction  is  a  known  and  continuous  function 
(termed  the  “reaction  function”)  of  the  stated  prediction.  Mid 
that  the  predicion  itself  has  finite  and  real  upper  and  lower 
bounda.  The  authors  conclude  that  the  possibility  of  correct 
public  prediction  exists  whenever  correct  private  predictions 
can  be  made,  and  that  the  very  real  diSculties  In  the  Utter 
sphere  are  “unrelated  to  their  potential  refiexivity.”  See  Emile 
Crunberg  and  Franco  Modigliani  (78). 

"On  the  dangers  of  allowing  projeetions  to  be  reduced  to 
mere  rationalixadons  of  current  policies,  see  the  comments  of 
Gerald  Colm  (28) ,  pp.  SSL 


thoMS.  The  more  plausible  of  the  resultant  outcomes 
might  then  be  regarded  as  a  confirmation  of  the  hypo¬ 
thesis  expressed  in  the  underlying  equation.  Similarly, 
any  projection  is,  in  its  purest  sense,  merely  an 
elaboration  of  the  consequences  entailed  by  a  specified 
set  of  assumptions.  As  such,  its  developmmt  must  not 
be  influenced  by  non-scientific  considerations.  In  shorty 
the  proper  but  crucial  role  of  value  considerations  in 
the  development  of  projections  is  not  too  different  from 
its  role  in  scientific  undertakings.  Initial  value  consid¬ 
erations  dictate  the  selection  of  assumptions  to  be 
expressed  in  the  projection,  just  as  they  dictate  the 
selection  of  problems  to  be  investigated  in  science. 
Then,  after  the  technical  work  has  been  carried  out, 
values  determine  how  alternative  projections  are  to  be 
identified,  used,  and  interpreted,  just  as  the  findings  of 
a  scientific  investigation  find  their  chief  practical  um- 
fulness  only  in  terms  of  the  values  they  help  to  advance. 
'Ihus,  the  proper  role  of  values  in  the  development  of 
projections  can  best  be  understood  by  recognizing  that 
the  most  important  purpose  of  long-range  projections 
is  to  express  our  values  within  a  framework  of  environ¬ 
mental  constraints  so  that  their  attainability  can  be 
assessed  and  realist'c  priorities  established.  However, 
the  projective  techniques  must  themselves  be  “value- 
free”  in  that  they  must  utilize  scientific  procedures  to 
the  fullest  possible  extent.** 

A  final  issue  in  regard  to  long-range  projections  is 
primarily  a  matter  of  policy,  though  it  carries  signifi¬ 
cant  philosophical  implications.  The  issue  relates  to 
the  question  of  public  versus  private  projections.  Once 
again,  the  fact  that  the  issue  arises  in  the  first  placj  is 
telling  evidence  that  projections  cannot  be  regarded 
solely  as  scientific  or  academic  exercises,  since  one  of 
the  hallmarks  of  science  is  the  free  dissemination  of 
findings.  But  a  projection  is  always  in  some  sense  a 
model  of  a  possible  or  probable  future.  Its  publication, 
especially  by  a  government  agency,  inevitably  carries 
with  it  the  authority  of  an  official  pronouncement,  with 
at  least  some  connotation  of  administrative  approval. 
It  is  therefore  tempting  to  differentiate  between  pro¬ 
jections  designed  for  internal  use  only  and  public  fore¬ 
casts  to  be  circulated  at  large.  One  advantage  of  such 

"Thit  position  in  regaid  to  the  role  of  values  in  behavioral 
science  is  a  familiar  one.  For  a  brief  exposition,  see  Richard 
Ruoner  (156).  In  economics,  the  need  to  supplement  Nalue- 
flee”  techniques  with  value-inspired  dehnitions  of  goals  is  well 
set  forth  by  Adolph  Lowe  (114).  Also  see  Lowe's  more  recent 
work  (IIS),  pp.  160  and  313.  For  an  excellent  critical  discus¬ 
sion  of  Lowe’s  thought,  see  Robert  L  Heilbroner  (87).  Philipp 
Frank’s  insightful  discussion  of  the  ‘‘extra-scientific"  purposes 
which  can  be  served  by  scientific  theories  is  highly  pertinent  to 
this  issue.  See  Philipp  C.  Frank  (56). 


a  differentiation  is  that  it  permits  the  technician  the 
free  exercise  of  his  talents  in  working  out  the  impli¬ 
cations  of  a  wide  range  of  alternative  assumptions 
without  the  onus  of  having  to  explain  or  defend  ffie 
government’s  interest  in  these  alternatives.  Under  such 
a  policy,  the  only  published  reports  would  be  forecasts, 
rather  than  projections.  They  would  reflect  some  kind 
of  plausible  compromise  between  the  goals  a  given 
administration  might  hope  to  attain  and  the  most  likely 
outcome  in  the  absence  of  govemmentally-inspired 
efforts.  As  such,  they  would  resemble  the  judgmental 
forecasts  of  the  National  Planning  Association,  but 
without  their  alternative  target  and  present  policy 
projections. 

Our  objection  to  such  a  policy  rests  first  upon  our 
basic  aigument  that  projections  should  provide  a 
public  awareness  of  possible  choices  or  alternatives  and 
their  probable  consequences.  A  single  forecast,  what¬ 
ever  its  ultimate  accuracy,  tends  to  create  an  atmos¬ 
phere  of  false  determinism  which  discourages  critical 
reflection.  One  should  note,  in  passing,  that  this  effect  is 
as  likely  to  be  found,  and  as  dangerous  in  its  implica¬ 
tions,  within  the  government  itself  as  well  as  among  the 
general  public. 

A  second  objection  relates  to  the  needs  of  the  policy¬ 
maker  himself.  A  single  forecast  is  of  course  required 
by  him  for  use  in  the  formulation  of  plans  and  pro¬ 
grams.  However,  the  policymaker  must  also  estimate 
the  effort  required  to  attain  particular  goals,  and  he 
must  consider  alternative  paths  toward  these  goals.  For 
instance,  he  needs  to  know  not  only  that  the  national 
economy  should  be  prepared  to  offer,  say,  80  million 
jobs  to  its  workers  by  1970,  but,  in  addition,  whether 
that  goal  can  be  reached  by  means  of  a  continuation, 
relaxation,  or  intensification  of  current  growth  rates. 
In  addition,  the  methodological  requirements  of  the 
newer  Planning-Programming-Budgeting  systems  with 
their  cost-benefit  accounting  schemes  demand  a  system¬ 
atic  comparison  of  program  targets  and  their  achieve¬ 
ment  costs  with  probable  outc^-nes  and  related  costs 
in  the  absence  of  such  programs.  Hiis  requirement 
can  only  be  met  by  the  development  of  alternative 
projections  and  a  publication  policy  which  gives  these 
alternatives  equal  publicity.** 

*‘The  n?ed  for  alternttive  projectiont  in  canTing  out  oaat- 
bvnefit  analyaea  is  brought  out  in  three  articles  by  Abraham  S. 
Levine  (109,  IIO,  III).  The  dangers  of  an  exclnaive  reUtaee 
upon  cost-benefit  analyses  in  determining  the  (eaaability  of  pro¬ 
posed  policies  in  the  manpower  field  are  well  stated  by  Robert 
Lekiicnman  (106).  For  a  balanced  statement  of  the  naes  of  coat- 
benefit  accounting  in  evaluating  manpower  policies  and  pro¬ 
grams,  see  Richard  A.  Lester,  op.  ctr.,  chapter  7. 
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UUmateljri  the  argaineat  favoring  the  free  pnblice- 
of  ifl  piojeotioiis  deemed  woifhy  of  preparation 
in  tlw  firat  phwe  reaU  cm  the  notion  that  an  infonned 
ptAlie  ie,  in  die  loag  ran,  our  beat  aafeguard  againat 
,dw  pwanit  of  ouwiae  (dijectivea  or  the  adoption  of 
buq^IMropriate  meant  toward  deaired  enda.  Paiticnlarly 
in  regard  to  the  fitinra,  the  public  needa  to  reoogniae 
diet  the  growth  of  acieoee  and  technology  doea  not 
ingdy  an  incieeae  in  determmum  but  rather  an  in> 
creeae  in  the  range  end  aoope,of  the  ehmutivet  before 
na.  If  projectiona  are  freely  developed  and  publithed  in 
ell  their  diveraity,  they  will  perhape  eerve  their  moat 
vital  fnnction  in  educating  the  public  to  a  conaciout 


awareneaa  of  ita  own  ineacapable  freedom  -the  free¬ 
dom  to  “invent  the  future.”** 

"Thia  tempdaf  phrate  ia  the  title  of  a  pnmieathe  eeaar  by 
Deimia  Gabor  (5'/).  Uaiag  Jomrenel’a  apt  terminology,  Daaiel 
Bell  atatea  the  iaaue  tety  aharply  ai  a  choice  between  deeUen- 
making  by  aeraglio  or  forum.  ^  Daniel  Bell  (11).  Compare 
the  remariu  of  Alfred  Sauvy  (op.  eit,  1962).  For  a  chalking- 
ing  diacuaaion  of  the  potential  dangera  in  the  achtevement  of 
reliable  prediction  ia  economlea,  aee  Gregor  Sebba  (163),  la  Ua 
modeat  and  temperate  commentary  on  Sebba’a  article,  Carl  F. 
Chriat  auggeata  tLat  the  danger  ia  not  on  the  imomdiate  horiaon, 
a  leaat  For  a  profoundly  peaeimiatic  analyab  of  the  naWeraal 
trend  toward  greater  rationalization  and  control  over  the  futnre, 
•ee  Roderick  Seidenberg  (164).  Seidenberg,  like  HetbeR 
Spencer  a  century  earlier,  foretella  a  world  which  haa  achieved 
total  predictability  at  the  coat  of  aurrendering  the  laat  veadgm 
of  meaningful  individual  choke. 


III.  THE  EVALUATION  OF  FROJEaiONS 


Both  the  improvenent  of  projections  and  the  de- 
tanninntion  of  their  status  as  forecasts  or  policy 
inatmments  depend  upon  the  criteria  wherdiy  they  are 
evaluated.  Such  evaluation  cannot  be  confined  to  the 
aiin|de  question  of  thmr  accuracy  because,  in  the  first 
place,  ]wojections  often  serve  other  functions  than  that 
of  prediction,  and  secondly,  the  evaluation  of  a  pro¬ 
jection  cannot  always  await  the  actual  occurrence  of 
tile  events  it  seeks  to  foretdO.  The  following  discussion 
distinguishes  four  areas  on  which  evaluation  of  pro¬ 
jections  may  be  focussed:  Their  assumptions,  their 
methodology,  their  outcome  or  content,  and  their 
uaefulneas,  given  thur  mtended  purposes. 

A.  The  Evaluation  of  Auumptiom 

As  previously  noted,  the  moat  glaring  deficiency  of 
moat  projections  is  the  insuflkient  specificity  of  their 
stated  assumptions.  Kids  suggests  the  primary  critmrion 
to  be  employed  in  evaluating  the  assumptions:  Are  they 
sufficiently  eqiUcit  to  permit  an  adequate  understand¬ 
ing  of  their  implicationa?  This  is  of  course  a  matter 
of  degree.  Insofar  as  any  projection  reflects  human 
judgment,  it  is  bound  to  incorporate  a  com]^  inter- 
rdation  of  factors  vdiich  cannot  be  fuDy  eapressed  or 
weighted.  Even  ia  the  case  of  the  more  obvious  judg¬ 
ments,  such  as  the  selection  or  rejection  of  a  particular 
variable  for  a  particular  equation,  a  detailed  exposition 
of  dm  nasona  for  the  decision  might  well  exceed  the 
espadty  of  both  the  technician’s  memory  and  the 
readm’s  powers  of  absorption. 

Furthermore,  the  practical  value  of  a  fuD  explanation 
of  the  assumptions  anderiying  a  given  projection  can 
be  questioned.  In  theory,  such  full  disdosure  would 
permit  a  ftdthful  replkmtion  of  a  projection.  What 
tiian?  Does  the  ability  to  duplicate  a  projection  affect 


ita  worth?  Replication  is  of  course  useful  in  disclosing 
mechanical  errors,  but  it  does  not  affect  the  plausibility 
of  the  underlying  assumptions.  Unfortunately,  it  is 
precisely  the  more  subtle  features  of  an  eiqiert’s 
judgment  which  are  most  difficult  to  express  or  dn|di- 
cate,  so  that  the  demand  for  totally  explicit  assumptions 
amounts  to  a  demand  that  experts  should  utiUae  only 
their  most  pedestrian,  readily  quantifiable  talents  in 
making  projections. 

The  assumptions  which  underly  a  projection  can 
also  be  ^aluated  in  terms  of  their  internal  consistency. 
For  exsi^iple,  a  labor  force  projection  which  poatulates 
a  continuation  of  military  expenditures  as  a  constant 
proportion  of  total  output  would  be  inconsistent  with 
the  assumption  of  a  drastic  decline  in  the  sise  of  the 
Armed  Forces,  unless  accompanied  by  the  further  stip¬ 
ulation  of  increasing  automatio.  in  military  tedi- 
nology.  Obviously,  this  evaluation  for  consiatesKy 
requires  a  certain  specificity  in  the  stateroenl  f  the 
assumptions.  “Global”  statements  rdating  to  the  as¬ 
sumed  absence  of  worldwide  conflict  or  depreasion  arc 
of  little  value,  since  they  permit  of  a  wide  range  of 
plausible  interpretation. 

A  third  criterion  for  the  evaluation  of  assumptions 
is  that  of  plausibility.  This  criterion  must  be  ap|died 
with  caution,  because  with  projections  in  particular, 
what  is  deemed  plausible  tends  to  be  greatly  inflneeioed 
by  what  is  regarded  as  desirable.  Furthermore,  the 
expressed  purpose  of  a  projection  may  be  to  speO  out 
the  implications  or  consequences  of  a  highly  iaqtrrdi- 
able  set  of  circumatances.  Such  an  exercise  may  he 
valuable  hsuristically,  and  should  not  be  criticinad 
because  the  stated  assumptions  are  unlikely  to  oecur 
in  fact.  Nevertheless,  given  the  limited  resources 


generally  available  for  thU  work,  the  bulk  of  projec¬ 
tions  must  be  designed  to  provide  a  reasonable  picture 
of  potential  realities,  so  that  the  plaiuibility  of  their 
assumptions  remains  an  important  consideration  in 
their  evaluation.  Jince  we  are  concerned  here  with 
making  judgments  about  other  judgments,  it  is  diffi¬ 
cult  to  establish  definite  limits  beyond  which  a  projec¬ 
tion  may  he  rejected  as  implausible.  However,  the 
attempt  is  worthwhile,  if  only  to  force  the  technician  to 
defend  the  adoption  of  assumptioiu  which  appear  to 
fall  outside  the  range  of  plausibilit}’.''* 

B.  The  Exxduation  of  Methodology 

The  evaluation  of  a  projection’s  methodoloby  does 
not  differ  from  that  of  any  scientific  procedure.  Such 
svaluation  should  consider,  first,  the  technical  accuracy 
of  the  results  obtained;  second,  the  apprupriatenesa  of 
the  analytical  methods  employed;  third,  the  adequacy 
of  the  data  and  their  treatment;  and  finally,  the  validity 
of  the  resnita  and  of  their  interpretation. 

Considerations  of  technical  accuracy  comprise  both 
the  mechanical  or  arithmetical  accuracy  of  the  pro¬ 
jection  it*  'If  and  the  correctness  of  the  definitions  and 
categories  of  data  that  were  utilized.  While  these 
matters  are  obviously  the  responsibility  of  the  tech¬ 
nicians  themselves,  independent  review  by  potential 
users  nuy  often  disclose  discrepancies  or  errors  which 
might  otherwise  escape  detection.  A  frequent  problem, 
particularly  where  time  series  are  being  used  exten¬ 
sively,  is  that  of  definitional  changes  or  changes  in 
coverage  which  impair  the  comparability  of  major 
categories  of  data  through  time.  A  related  problem  is 
the  need  to  fully  explain  the  adjustmmits  to  the  data 
which  may  have  been  deemed  necessary  Such  adjust¬ 
ments  are  frequently  made  in  haste  and  inadequately 
recorded,  ao  that  analysts  wrrking  with  long  historical 
eerier  of  data  are  commonly  unable  to  determine  the 
nature  and  magnitude  of  the  •adjustments  that  may 
have  been  made  at  the  time  the  data  were  being 
originally  processed. 

With  respect  to  the  adequacy  of  the  methods  used, 
it  is  necessary  that  they  be  considered  not  only  in 
comparison  with  possible  ahematire  methods,  but  also 
in  terms  ot  the  time  and  resources  available  to  the 
technician.  The  ease  with  which  the  inclusion  of  ad¬ 
ditional  variables  or  the  use  of  ahemative  methtrds 
may  be  suggested  is  not  matched  by  the  ease  with  which 
theae  elaborations  can  be  carried  out.  Where  govern¬ 
ment  agencies  have  presented  a  single  forecast  rather 
than  an  elaborata  set  of  projections,  they  have  com- 

"See,  for  extmph,  Hioibsi  Denbery  sad  sarocisle*  (op.  eii., 
1966)  sod  itw  critics!  review  by  Sopliia  Cooper  sad  Pesit  F. 
Johowoa  is  the  esaw  btac,  pp.  69-7B. 


^  monly  done  so  because  of  Binited  resources,  nnd  not 
because  of  a  deliberate  policy  of  restricting  puMkation 
to  a  single  series.  GiveB  these  inevhabk  constraints, 
however,  the  technician  always  faces  a  dioioe  between 
better  or  worse  methods,  and  a  review  of  his  mediod- 
ology  is  therefore  apprqrriate  in  any  case. 

Two  further  coosideratioiM  must  be  stressed  in  the 
evaluation  of  the  trrethoda  enqdoyed  in  projectioae:. 
The  appropriateness  of  the  methods  givev  the  quality 
of  the  avaffable  data,  and  ffieir  suitability  in  terms  of 
the  purposes  the  projections  are  intended  to  serve.  It 
is  not  necessarily  incorrect  to  enqdoy  dabonUe  methods 
of  analysis  with  limited  data  or  data  of  poor  quality. 
In  fact,  such  data  nuy  reqttire  the  land  of  careful 
tailoring  which  such  methods  can  provide.  However, 
a  common  effect  of  daborate  methodology  is  to  convey 
a  sense  of  greater  predskm  than  exists  in  fact,  and  to 
conceal  the  deficiencies  of  die  data  beneath  an  im¬ 
pressive  scaffolding  of  quantitative  analysia.  There  are 
no  simple  safeguards  against  the  misnae  of  statistical 
methods  other  than  the  alow  inqirovenient  in  the 
methodological  sophistication  of  technicians  and  users 
alike.  One  hdpful  device  is  the  roodiw  application  of 
simple  measures  relating  to  the  acenraey  of  the  data 
(where  such  measures  can  be  devised).  It  is  now 
standard  practice  to  show  sneh  measures  akatgaide  the 
pertinent  statistics  in  all  statiWical  publications.  The 
publication  of  equivalent  meamues  in  connection  with 
projected  values  would  alert  both  the  teefanidsn  and 
the  user  to  the  limitations  of  the  data  and  might  warn 
them  against  drawing  unwarranted  inferaices  from 
the  given  results.*'^ 

C.  The  Evaluatio:*  o'  CoHteMt 

Hie  actual  results  of  a  projection — the  levds  and 
distributions  which  ore  shown  for  future  dates — must 
themselves  be  evaluated  direedv,  opart  from  couridma- 
tions  of  assumptions  and  methodology.  This  evolortion 
may  be  carried  out  in  three  directiona.  First,  thoe 
is  the  question  of  the  conaktesicy  of  the  outcome  wiUi 
the  stated  assumptions;  this  b  a  separate  issne  from 
that  relating  to  the  plausQwlity  of  tfr)  sasumptions  ao 
such.  If,  for  example,  a  labor  force  projectioo  asauines 
a  continuation  of  early  marriage  and  childhearing 
practicee,  a  major  rise  in  the  projected  labor  force 
participaiion  rates  of  young  adak  woesan  ahould  at 
leeat  be  questioned.  A  secosMi  direetian  of  inquiry 
coocemt  the  ability  of  the  projection  to  sutiify  dw  pur¬ 
poses  for  which  it  was  developed.  If  its  asahi  purpoae  b 
to  provide  •  puhitc  predirtion  or  forerert,  the  publka- 

*Aa  Bo(«d  prrviomir,  lUs  k  a  sriwery  rw—  lailittiia  af 
Oikar  MorgvMcrs,  On  ikt  Accmrmtj  ni  CesMeke  (MstraaSMea, 
p.  302ff. 
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lisa  «f  ■  nqfi  of  fahm  nfleoting  ahenuliTo  mIs  of 
MMIllieM  wiD  not  rafke,  noloM  one  of  the  aoriu  k 
fcWnHfced  ao  die  mok  likdy  or  jodgmenUl 
WlwiHoa,  Qa  Ae  oAer  bond,  if  ■  pcojectioo  k  deoignod 
lo  provide  wHiMke  oi[  die  probeUe  fanpoet  of  akeme* 
dvo  poikka,  it  aboidd  iadkate  tba  different  effects  of 
daaa  poBcka  bjr  means  of  separate  projectkma  reflect- 
bi|  eoeh  of  dke  akemathes. 

A  aion  Afkak  jaobleni  in  evaluatioa  of  coiaaBt  k 
to  determine  dm  amonist  of  detail  and  aocorsey  that  k 
boA  snffckat  and  neeenary.  Tbe  needs  of  tbe  naer 
map  ofkn  be  met  by  a  shgk  nnmber — ^for  example,  be 
may  raqnire  a  projection  of  tbe  popolstion  of  tbe 
eooatry  k  dm  year  2000.  However,  toch  a  figure  can 
beat  be  obtained  by  working  with  individual  ageoez 
eobortS)  and  aggregating  the  results  as  a  final  step. 
Unis,  dm  amount  of  detail  required  in  developing  a 
iwojeetkm  k  more  a  function  of  tbe  methodology 
employed  than  the  purpoee  of  the  projection  hadf.  Tbe 
real  queation  k  therefore  whether  the  results  of  a  given 
projeedao  reflect  an  optimam  compromise  between 
what  tbe  availaUe  data  can  offer  and  the  detail  required 
to  serve  the  purpose  of  the  f  Tojection.°* 

A  third  direction  of  aval  ladon  of  content  leiatea  to 
dm  reoognitioa  given  to  toe  band  of  uncertainty  or 
error  wbkb  aurrouzids  any  projection.  Above  all,  pro¬ 
jections  should  avoid  giving  the  impression  of  greater 
predaion  or  foreknowledge  than  is  warranted.  This  is 
only  partly  a  queation  of  semantics.  One  can  argue  that 
a  projectkm  k  not  a  prediction,  and  that  a  reading  of 
tbe  text  win  invariably  dkdoae  a  convenient  eacape 
dauae  in  tbe  form  of  tbe  usual  **oeteris  paribus” 
proviso,  and  that,  in  any  event,  'iet  tbe  user  beware.” 
However,  in  the  practical  world,  forecasts  are  used  as 
given,  and  the  attendant  qualifications  tend  to  drop  out 
of  consideration.  Thk  danger  can  only  be  minimised 
by  ^ving  equal  prominence  to  tbe  estimated  band  oi 
mwertainty  eafroanding  a  given  projection  es  to  tbe 
lusnka  dmnmeives.** 

D.  ffnafsatiag  At  Vttfmktu  of  Froftctioiu 

Becnaea  projectkwa  are  typically  developed  for 

*€sesMirslteas  sf  aessrscy  five  rtes  is  llw  qiwnios  of  iW 
islsrtoi  esM  of  sa  sosrsssiwit*  wsat  as  osifarsiiiaMti.  For 
SaffaMo  ti  ansfiiwrr  sisssiM.  **  saJorswisnu  awy  pnrt 
•tan  eaaiir-  fk  Ukis  s*iat.  sas  iW  raaaHu  of  SUary  SoaraUoai 
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practical  pnrpoaea,  thdr  evshiation  must  include  some 
oMisiderstion  of  their  nsefuhwne  h  meeting  tbeee 
purpoees.**  Since  many  projections  serve  •  vsricty  of 
usee,  such  evaluation  reqnires  some  notion  of  prioritiee. 
When  projecdoim  are  designed  primarily  to  serve  tbe 
needs  of  the  poUcymaker,  their  evaluation  must  recog¬ 
nise  tbe  constraints  imposed  upon  tbe  technician  by  tbe 
preeeing  demands  of  policy  formulation.  The  basic 
function  of  tbe  poUcymaker  k  to  map  a  course  of  action 
into  an  nnoertsin  future.  He  k  not  merely  called  upon 
to  make  prognostications  from  time  to  time;  he  k 
immersed  in  tbe  future  all  tbe  time,  and  most  constantly 
formukte  and  express  hk  outlook  on  tbe  future  in 
rdstion  to  specific  objectivm  end  policiee.  He  must 
therefore  be  provided  with  a  variety  of  statements 
having  a  future  reference,  but  related  to  current  trends 
and  policies.  Tbeae  atatements  must  often  be  prepared 
on  very  short  notice,  and  may  commonly  be  altered 
quickly,  with  Uttle  chance  for  more  than  the  most  super¬ 
ficial  research.  Hence  tbe  evaluation  of  such  statements 
or  forecasts  cannot  be  carried  out  in  terms  of  the  tbor- 
oughnets  of  tbe  methodology  and  underlying  research, 
but  must  consider  instead  tb«r  timeliness  and  over  all 
consistency.” 

The  simplicity  of  a  projection  technique  may  itself 
be  an  important  aaaet  to  the  poUcymaker,  since  be  is 
often  required  to  convey  some  notion  of  the  assump¬ 
tions  underlying  his  forecast  briefly  and  in  nontechnical 
language.  Projections  designed  to  meet  such  purposes 
should  not  be  evaluated  in  terms  of  criteria  appropriate 
to  tbe  analysis  of  the  more  recondite  research  efforts  in 
the  field. 

In  contrast,  projections  designed  to  provide  the 
policymaker  with  detailed  guidelines,  measures  of 
progress  toward  a  specific  goal,  or  elaborate  ddinea- 
tions  of  the  possible  consequences  of  ahemative  coursm 
of  action  must  be  evaluated  by  considering  their  as¬ 
sumptions.  methodology,  and  overall  consistency  in 
the  fullest  detail.  Such  projections  should  open  up 
options;  their  assumptions  must  be  sufficiently  explicit 

"A*  aoted  prcTMinily.  ike  priaeiptc  porgoec*  of  f«v«mawMtl 
prai<-:iioiH  are  sdmirsbly  •amaurited  by  Jamc*  W.  KnowW, 
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b  toftkeominf.  Sao  Jscok  Miaocr  a»d  Victor  ZenMsdia  (iJi). 
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to  permit  a  clear  diatinction  among  the  ahemative 
aeriea.  Their  evaluation  ahonld  therefore  focus  on  the 
ezpLcitneM  of  the  aaaumptiona,  the  clarity  with  which 
their  likely  consequences  are  descrihed,  and,  moat  im> 
portant,  their  variety  and  imaginative  composition. 

Projections  which  heat  fnlfiS  the  above  purposes 
must  actually  serve  in  two  capacities,  and  should  there* 
fore  be  evaluated  accordingly.  Their  overt  function  has 
already  been  emphasised — to  alert  the  policymaker  to 
the  scope  and  range  of  the  ahematives  before  him  and 
offer  added  insight  into  the  attainability  of  desired 
goals,  or  the  desirability  of  attainable  ones.  Their  hid¬ 
den  function  is,  however,  just  as  important— to  provide 
an  awareness  of  the  limitations  to  action  which  exists 
in  a  given  situation,  the  constraints  imposed  by  limited 
resources,  political  exigencies,  the  weight  of  habit  and 
custom,  the  occurrence  of  unpredictable  events,  and  the 
like.** 

As  was  discussed  in  the  second  chapter,  projections 
serve  a  number  of  purposes  additional  to  those  men¬ 
tioned  above.  Following  the  sequence  of  puiposes  sum¬ 
marized  there,  appropriate  criteria  for  their  evaluation 
can  be  suggested  in  each  case.  First,  the  ability  to 
provide  advance  warning  of  impending  strains  or  im¬ 
balances  given  a  continuation  of  current  trends  affords 
a  stringent  criterion  for  evaluation.  The  evaluation  of 
a  projection  on  the  basis  of  its  success  in  this  respect 
necessitates  an  ex  post  facto  analysis,  since  it  presup¬ 
poses  knowledge  of  what  did  or  did  not  occur  over  the 
projection  period.  Even  with  such  knowledge,  this  kind 
of  evaluation  is  difficult.  It  is  always  possible  to  detect, 
with  the  advantage  of  hindsight,  signs  of  impending 
events  which  were  not  at  all  obvious  prior  to  their  oc¬ 
currence.  Furthermore,  forecasts  which  turn  out,  by  the 
merest  chaiKe,  to  be  remarkably  accurate  are  likely  to 
convey  to  both  the  forecaster  and  hb  naethods  an 
undeserved  reputation  of  forecasting  ability.  The  repu¬ 
tations  of  many  traditional  soothsayers  and  oracles 
were  largely  buih  up  in  this  soanner.  Since  their  prog¬ 
nostications  srere  commonly  expressed  in  sufficiently 
ambiguous  terms  as  to  be  coasbteat  srith  a  sride  variety 
of  outcomes,  their  forecasting  ability  b  not  surprising. 
Today's  qusntilalive  forecasts  do  not  usually  allow  such 
generom  boun^  for  forecasting  error. 

In  evalustiitg  projectioiM  on  the  bssb  of  their  ability 
to  provide  advance  warnings  of  coming  change,  it  b 
necessary  to  recall  their  "seH-denyiag’'  poestbilities.  A 

"Tba  ntMtkt  of  Siiaee  Kanwt*  prsri^t  •  rWar 
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projection  of  an  impending  problem  which  sets  b 
motion  a  course  of  action  which  overcomes  the  problem 
will  have  served  a  vital  function  ahhong^  it  tuma  out 
to  be  highly  inaccurate. 

Evaluatbg  the  ability  of  projeetiona  to  elucidate  the 
probable  conaequenoes  of  ahemative  policiea  or  pro¬ 
grams  requires  some  decision  as  to  the  appn^riate 
limits  of  the  technician’a  professional  competence.  For 
example,  current  laher  force  projections  ahow  tuhban- 
tbl  brreaaes  in  the  labor  force  peiticipation  rates  of 
young  adult  women.  Should  such  prajectiona  be  accom¬ 
panied  by  a  dbeuseion  of  the  possible  unintended  cou- 
sequence  of  such  a  trend,  such  a«  s  possible  rise  b 
juvenile  dclbquency  or  other  strains  on  the  stiticture 
of  the  family?  **  Hie  fact  that  such  speculations  cut 
across  conventional  disciplinary  boundariea  does  not 
affect  their  potential  usefulness  to  the  policymaker.  In 
the  lohgrun,  the  noneconomic  consequences  of  particu¬ 
lar  manpower  policies  may  turn  out  to  be  even  more 
significant  than  their  economic  impact. 

A  far  more  stringent  criterion  for  the  evaluation  of 
projections  b  the  consideration  of  their  success  b  as¬ 
signing  probabilities  tc  the  possible  outcomes  of  speci¬ 
fied  assumptions  or  skerastive  policies-  Only  strictly 
quantitative  projections  offer  even  the  possibility  of 
meeting  thb  requirement  with  any  precision.  However, 
it  should  be  poaaible  for  technicians  who  develop  alter¬ 
native  series  of  projections  to  accompany  them  with 
some  kind  of  rough  probability  ranking,  arrived  at 
judgroentally,  on  a  scale  from  h'ghly  probable  to  highly 
improbable.  The  use  of  such  a  scale  would  also  be 
helpful  b  selecting  a  single  roost  likely  series  for  fore¬ 
casting  purposes.** 

Where  projections  are  designed  simply  to  extrapolate 
trends  from  a  specified  period  in  the  past,  their  evalua¬ 
tion  b  more  straightforward.  Here,  the  relevant  criterU 
are  the  adequacy  of  the  ststemoit  of  assumptions,  the 
appropriateness  of  the  technique  sdected  to  represent 
the  central  tendsney  of  the  observed  data,  and  the 
accuracy  of  the  derived  extrapolation  itself.  Similar 
criteris  are  appropriate  ia  evduating  projerliorts  de¬ 
veloped  to  prescribe  ahemative  paths  from  a  fixed  base 
line  to  s  predrtermiaed  objective  or  target  level.  The 
adequacy  of  such  projections  reato  ukimstely  on  their 
coosbterxr  with  the  specified  assumptions. 

"Tki*  b  uUM>d»d  to  b*  parefr  iOealtstb*.  ssci  pnsappeat* 
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Tbe  inenMiag  aw  of  projocdon*  t»  dement*  in  « 
largtr  modal  or  is  a  daio  of  related  projectioos  sug- 
gmla  fardtar  criteria  for  timir  evaluation.  An  obviona 
tumridaralioii  la  tba  nwtuai  cooaiatenqr  of  tlte  uaump- 
tfoaa  vridob  tadmijr  aadi  of  dm  projection*  which  kre 
Hahad  tefothar.  la  preparing  aaj  projection,  *ome  at- 
teadoB  dioaM  be  p^  to  the  more  obviou*  uaagee'it 
wfll  bo  givan  ia  amodetion  %ridi  rdated  projection*. 
For  eBaaiple,  a  pu|»alatioa  projection  i*  more  uaeful  os 
a  bane  for  labor  fona  projectioa*  if  it  provide*  greeter 
age  detail  for  poraon*  age  16  to  24  and  55  and  over, 
afatee  the  woricar  rate*  of  peraon*  in  theae  ages  change 
rapidly.  By  the  aame  toten,  the  aaefulneis  of  population 
projections  would  Le  greatly  enhanced  if  •quunte  pro¬ 
jections  coold  he  ihown  of  the  chriBan,  noninathutional 
popnlatioo,  for  me  with  projected  civilian  labor  force. 
Sfaniiariy,  projectkms  of  total  and  civilian  labor  force 
ihonld  be  made  available  to  meet  different  user  need*. 
AUmeigh  the  tailorfog  of  a  projection  to  meet  special 
requirement*  of  this  sort  may  add  considerably  to  ite 
coat,  the  ad  hoc  adjnstmenta  which  must  otherwise  be 
made  by  the  aaer  hinwelf  are  likely  to  be  both  costlier 
and  lam  aocarate. 

Moat  diScnk  to  evaluate  are  du  projections  which 
offnr  the  greatete  pntenrial  challenge'  those  deliberate 
invenrinm  or  scmarins  of  d»  foture  which  seek  to 
explore  &a  oanasanaBoas  of  a  set  of  hypothetkal  cir- 
rametanuin.  fVehsps  the  greateat  danger  in  this  kind  of 
imeginerivH  riarciaa  is  the  non  sequitur.  However,  we 
are  daiiiag  base  with  possibilities,  not  with  neceasitiaa, 
ao  that  s  weaker  criterior  of  evalnation  than  strict  im- 
;dication  is  caBed  for.  IWbaps  we  are  again  left  with 
plansihiKfy  as  the  only  criterion  which  affords  a  basis 
for  criticimi  wbils  rsoogniaiag  that  any  conceivable 
falare  is  possible  in  a  nondetenninistic  onirorse.  The 
prnhleni  srith  phnsibility  is  that  k.  too,  demands  sn 
operatiooally  maaningfsd  dribiitioa:  otherwise,  it  re¬ 
flects  Iktle  asort  than  ana’s  cnhnral  background,  values, 
end  psedilsrtinns  Siaoe,  to  tbe  ordinary  critic,  no 
foture  is  pisnsihis  uniam  h  rsffeols  an  idealisation  of 
the  prestnt.  tha  trkirims  of  hauriitic  projectie.'w  of  the 
grounds  of  their  plaoribiiky  srould  iiriroduce  s  power¬ 
ful  cowssrvstivs  biaa  inie  their  ershiafion  and  thus 
greatly  waalren  dwir  potaatral  contribution.  A  bctto' 
has**  for  mrslnstinn  assy  be  found  in  considering  the 
theughtfolassB  and  inguanky  of  their  dereiopwant  and 
the  avuriB  adsqnacy  ol  the  structure  of  their  hypo- 
thatiral  dsSanaiaanti  mi  ossMsquencas.** 

Tha  svalusHnu  of  psojartions  in  lertm  of  their  ability 

*S**  dW  ns-shoBl  steerMSisa*  st  lt»wrsw  CmM  (71).  Far 
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to  fulfill  a  public  information  function  brings  into 
consideration  their  potential  audience.  As  previously 
noted,  tbe  requirements  of  technically  trained  users 
differ  markedly  from  those  of  either  the  policymskerv 
or  the  general  public.  Both  mohipie  projections  and 
single  forecasts  are  apt  to  mislead  the  general  public; 
tbe  former  because  of  their  indefinite  quality  and  tbe 
latter  because  of  tbeir  apparent  determinism.  In  addi¬ 
tion,  a  forecast,  when  published,  carries  the  authority 
of  the  publishing  agency  behind  it  ai.J  tends  to  be 
interpreted, 'in  some  sense,  as  agency  policy.  For  these 
reasons,  it  is  neceuary  to  evaluate  both  projections  and 
forecasts  in  terms  of  the  balance  they  maintain  between 
tbe  theoretically  potsible  and  the  historically  probable. 
While  both  projoctiona  and  singular  forecasts  are 
rooted  in  historical  trends,  the  former  can  only  be  given 
an  appropriate  range  by  onphaaizing  theoretical  possi¬ 
bilities  representing  departures  from  past  trends.  Tbe 
forecast,  on  the  other  hand,  must  reflect  a  selection 
from  among  these  possibilities  with  emphas’s  upon  the  j 

historical  trend.  Unlike  projections,  forecasts  should  j 

not  be  expected  to  depart  radically  from  pak  trends  , 

unless  the  departure  can  be  snppo'ted  by  strong  | 

evidence.  The  burden  of  proof  in  a  forecast  (again, 
unUke  a  projection)  should  rest  on  the  assumption  of  i 

a  major  change  in  underlying  structure  rather  than  on  | 

a  continuation  of  past  trends. 

Tbe  primary  need*  of  the  general  public  in  regard  to 
projection*  and  forecasts  are  for  clarity  of  exposition, 
careful  labeiing  of  results,  and  an  understanding  of  the 
vital  educational  role  to  be  played  by  any  pronounce- 
mmte  about  the  future.  In  evaluating  projections  as 
public  doctunen*.*  therefore,  primary  consideration 
should  be  given  to  their  expression  in  nontechnksl 
language,  with  only  a  necesoary  minimum  of  expository 
detail  regarding  methodology  and  ttsum[^ion*  and 
greater  emphasis  on  purposm  and  uses.  This  require- 
romt  cannot  easily  be  met  wbsn  projections  are  tbe 
product  of  technically  complex  econometric  procedure, 
but  even  here,  ■  nontechnicft’  summary  of  their  as¬ 
sumptions  and  procedures,  f(>liowed  by  a  careful  state¬ 
ment  of  their  intended  uses  i*  essential  if  the  projec 
lions  are  to  be  used  intrlligenily  br  those  wh  i  Isck  the 
technical  training  of  the  econometrician.** 

To  the  informed  public,  forecMl*  are  coimnoi:ly  the 
object  of  snxmed  sceptkiem.  reflecting  the  common-  . 
sense  view  ihtf  a  forecaal  b  only  good  if  it  b  accui  ';te, 
and  ihb  quality  can  never  he  determined  beforehand. 
Furthermore,  forecasts  are  sridom  the  object  of  wide 
ipraed  pobbr  interea  at  the  time  they  afe  bmed ;  only 

•The  nmrt  rvisrtu  ol  Sorretsrr  Wirw  sf»  perthw^  ta  ihis 
nusnL  5w  Vttbni  W.  Vim  (30)). 
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when,  with  tlie  passege  of  time,  they  turn  oat  to  be 
glaringly  wide  of  the  nurk,  or,  more  rarely,  aurpria- 
ingly  close  to  reality,  do  they  receive  great  pnbUcity. 
Ill  .ii.'  latter  instances,  the  aura  of  certitude  which 
comes  to  surround  both  the  successful  technician  and 
the  succTiiaful  technique  may  be  more  harmful  than 
complete  scepticism,  since  it  promoter  slavish  imitation 
I  rather  than  innovation.  For  these  reasons,  the  educa¬ 
tional  purpose  to  be  served  by  the  publication  of  pro¬ 
jections  rather  than  forecasts  is  of  great  importance. 
Only  when  informed  individuals  understand  the  differ¬ 
ences  among  projections,  forecasts,  and  predictions  can 
they  be  expected  to  examine  projections  in  t!ie  light  of 
more  relevant  criteria  than  merely  their  predictive 
accuracy.  When  projections  are  recognised  as  accounts 
of  possible  futures  or  “futuribles”  (to  use  de  Jouvenel’s 
tern),  they  '•an  properly  be  viewed  os  possible  exten¬ 
sions  of  human  valu*^  and  goals  ( i.e.,  as  target  projec¬ 
tions)  or  as  extrapolations  of  trends  which  may 
!  threaten  these  values  and  goals.  Only  then  can  the 
j  expression  of  public  reaction  to  given  projections  serve 
to  inform  the  policymaker  of  the  public  will  in  regard 
to  the  essential  task  of  guiding  the  future  growth  of 
those  aspe'-'s  of  our  society  which  are  deemed  appro¬ 
priate  objects  of  national  planning. 

The  above  considerations  lead  us  to  conclude  that 
projections  cannot  be  adequately  evaluated  on  technical 
1  grounds  alone,  and  a  fortiori,  they  should  not  Le  evalu- 
f  ated  only  by  technical  experts.  Their  uhinute  signifi- 
I  cancc  ecu  only  l*e  perceived  within  a  breeder  frame  of 
reference,  comprising  the  society’s  goals  and  values, 

j  "Thr  danjerj  of  vlf-serring  gortromcnlal  secreer  in  ihu 
I  vilal  «rra  are  well  summarued  by  Alfrrd  .Sa^ry  (op.  cil.,  1962, 

j  p.  12-if).  \f  one  of  France’s  leading  denugrapbers,  SaicTT  hts 

I  had  calensive  experience  in  the  applkalioa  of  democraptuc  and 
j  ri-onomic  lechntques  in  li:-  devrlopraent  of  eeonoasic  policiea- 
For  a  liief  exposilion  of  hit  thinking  on  nemognpiuc-ecoBomic 
j  projrclioiu.  !«  .Alfred  Saury  (op.  cil.,  19S9). 
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and,  to  uae  Shackle’a  term,  the  current  “rig”  of  the 
aaciety~i.e.,  the  aet  of  tenaiona  or  preoaurea  ariaing 
from  gapo  which  exiat  at  a  given  time  between  the 
actual  situation  and  our  valuea  and  aqtirationa."  The 
construction  of  such  a  frame  of  reference  is  coming  to 
be  recognised  as  a  crucial  task  for  the  bdiavioral  scien¬ 
tists  during  the  remaining  third  of  the  present  century. 
Such  a  framr  of  reference  would  require  the  derdop- 
ment  of  a  system  of  social  statistics  supplementary  to 
the  economic  statistics  on  which  so  many  current  esti¬ 
mates  of  the  society’s  condition  must  rdy.  Given  such 
a  system,  selected  measures  could  be  used,  separately 
or  in  combination,  as  social  indicators — i.e.,  readings 
of  the  current  status  and  evident  trends  of  the  society 
in  regard  to  specified  goals  or  program  objectives.** 

The  primary  aignificarice  of  this  development  for 
projections  of  labor  force  is  that  it  affords  both  a  chal¬ 
lenge  and  an  opportunity  to  bring  the  problems  of  labor 
force  growth  and  manpower  development  out  of  the 
constricting  framework  of  a  conventional  supply- 
demand  calculus  into  a  broader  perspective  of  social 
goals  and  aspirati''ns.  Only  within  such  a  perspective 
can  the  potential  costs  and  benefits  of  alternative  man¬ 
power  policies  be  estiiruted  in  social  and  psychological, 
as  well  08  economic  tenna.^* 

•George  L  S.  Shackle  (f66>,  pp.  in-UR  ThU  essay  U  a 
perceptive  commentary  on  three  lectures  by  Bertrand  de  Joov- 
cncl  on  “model*  of  conjeett  re." 

"A  gen.-ral  outline  of  such  a  system  is  developed  by  Bertram 
Cross  (73). 

".A  challenging  arcouni  of  the  plurality  of  rationalities  which 
must  be  considered  in  the  development  of  social  policies  is  pro¬ 
vided  bv  Paul  Diesing  1 44) .  The  need  to  incorporsic  non¬ 
economic  factors  iii  the  solution  of  ecoisomic  problems  is  also 
well  state.!  by  .Allan  C.  CrTsckv  (75'.  Als*.  see  Melvin  L.  Green* 
hu!  (69)  ind  Alfred  L  Thimm  (196'  A  clear  statement  of  the 
Iranseconomk  nature  of  manpower  probleo’s  is  givee  in  Henry 
Dsvid  (.18).  Also  see  the  discussion  of  tl  •  ue  by  Hirold  L. 
Sheppard  i/67i. 
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Mr.  Irwin.  The  smart  bird  at  the  Bureau  of  Census 
pushing  all  those  buttons  is  smart  enough  not  to  kill 
the  dumb  bird;  keep  making  projections  we  want  to 
use.  I  don’t  know  if  thet  dumb  bird  is  quite  that  dumb. 
He  is  liable  to  get  the  idea  he  can  make  the  projection 
and  do  -.vithout  the  Bureau  of  Census.  I  am  personally 
sympathetic  with  the  idea  of  a  forecast  figure,  a  prefer¬ 
ence  choice.  I  think  the  tenor  of  my  remarks  indicaleil 
we  would  if  such  a  statement  is  made.  This  is  an  old 
argument  between  monitors  making  projections  and  the 
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judgment  calls  although  I  would  add  a  parenthetical 
comment :  We  represent  B  and  C  as  more  reasonable 
and  probable  and  A  and  D  as  extremes,  which  is  a 
minor  point.  I  should  have  elaborated  on  the  expecta¬ 
tions  and  stated  we  are  basing  the  choice  on  this  na¬ 
tional  survey  which  has  been  carried  out  three  times: 
in  1955, 1;960, 1965;  asking  women  how  many  children 
they  expect  to  have  in  5  years.  This  survey  goes  no 
further  than  5  years  and  the  data  has  remained  constant 
at  three,  knowledgeable  people  in  the  field  working  on 
the  expe^ations  data  pointed  out  we  were  talking  only 
to  young  women  of  childbearing  age,  we  don’t  inter¬ 
view  anyone  10  to  14  years  of  age.  Within  5  years  these 
girls  produce  a  number  of  babies.  If  we  get  a  range  of 
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probability  around  one  &g’'.re.  I’m  aure  we  would  do  it. 
I  !eel  we  ought  to  try. 

When  Doiis  was  showing  the  problems  which  beset 
him  with  the  female  participation  rate  2S  to  29, 1  would 
take  'Jie  oflScial  bureau  position,  the  one  he  har  pre- 
senteo.  He  presented  a  series  of  assumptions.  We  say 
they  are  bald  assumptions;  If  they  come  true  the  popu¬ 
lation  will  be  thus  and  so,  but  since  our  assumption 
becomes  his  starting  point,  with  the  mental  torment  we 
put  him  through  regarding  the  number  of  babiM  to  be 
produced,  which  series  to  select,  I  wonder  if  this  is  a 
useful  exercise.  Our  position  is,  “Look,  folks,  we  don’t 
Know  exactly  what  is  going  to  happen;  these  are  your 
alternatives.  You  have  certain  situations;  use  this. the 
way  you  want.”  Ihis  is  what  we  did  and  why,  which  is 
quite  involved.  I  feel  sympathetic  because  before  I 
came  to  the  Bureau  I  was  irritated  to  find  they  were 
giving  me  four,  not  three.  You  can’t  even  pick  the 
middle  one  and  I  had  to  read  this  long  detailed  ex¬ 
planation.  It  is  tough  and  1  am  not  really  presenting  a 
position  fo  much  as  asking  a  question. 

Dr.  JdHNSTON.  1  ended  by  saying  whatever  else  pro¬ 
jections  do  they  educate  us  and  I  don’t  say  it  as  pro¬ 
fessional  training. 

Dr.  Slafkosky.  What  do  you.suggest  as  the  proper 
alternative  criteria  for  projection.  Ycfii  are  saying  you 
can’t  use  the  fact  they  were  either  correct  or  wrong  in 
some  sense.  The  proper  criterion  sivould  be  internal  as 
I  understood  it.  You  stated  they  were  educational. 
Kindly  elaborate  as  to  the  kind  of  education  you  are 
referring  to. 

Dr.  Johnston.  I  don’t -know.  I  get  to  the  outer  edge 
of  my  own  ability  to  cOtnprehend  the  area  in  which  I 
earn  my  living.  We  are  groping  for  a  sense  of  social 
goals,  social  indicatcj^s  or  |)rogre88  toward  social  goals. 
There  was  a  book  published  last  year,  “Social  Indica¬ 
tors.”  As  my  old  professor,  at  Harvard  used  to  say. 
“Tne  only  thing  that  gives  this  book  unity  is  the  cover.” 
but  in  the  book  is  a  very  goM  cotlection  of  work  relat¬ 
ing  to  fhe  problem  of  the  social  s:;iences — how  do  we 
get  even  measures  to  our  goal,  much  less  the  goals  wc 
all  agree  on.  Wc  want  healthy  and  peaceable  people  all 
over  the  world.  We  would  like  people  to  be  educated, 
so  they  can  fulfill  their  creative  potentials.  All  these 
goals  I  state,  may  sound  clear  enough  but  they  are  not 
transparently  clear.  You  can  have  something  that 
sounds  clear  when  you  read  it  but  it’s  ambiguous  in  its 
implications  and  its  connotations.  All  of  our  social 
goals  are  ambiguous  in  a  sense.  What  do  we  mean  by 
health,  education,  peacefulness,  and  social  order.  We 
want  growth.  This  contradicts  the  notion  of  order. 
Maybe  you  can’t  get  growth  without  disorder.  We  need 
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to  know  when  we  have  a  goal  to  agree  on  and  how  we 
measure  our  progress  toward  it.  “Social  Indicators”  by 
A1  Biderman  has  a  chapter  relating  tc  the  Uniform 
Crime  Reports  of  the  FBI,  and  tries  to  show  how  mis¬ 
leading  the  statistics  are.  You  get  the  idea  they  do 
more  harm  than  good  in  informing  the  public  about 
the  phenomena  from  this  article.  I’ll  try  one  measure 
of  the  effectiveness  of  a  projection  to  my  way  of  think¬ 
ing:  Has  it  given  more  fruitful  directions  to  search  for 
measures  of  attainment,  progress  or  advance?  Does  it 
give  us  insights?  Does  it  open  possibilities  of  seeing 
relationships  we  would  otherwise  overlook?  Does  it 
open  possible  ahernativrs  we  would  not  have  consid¬ 
ered?  The  idea  of  giving  every  potential  Viet  Cong  a 
Ph.  D.  is  a  typical  far-out  stupid  irrelevant  remark,  but 
20  years  ago  this  might  not  have  been  unfeasible.  When 
political  relationships  were  sufficient  we  could  take 
some  of  the  leaders  of  these  underdeveloped  countries 
and  educate  them.  It  might  have  been  feasible  in  China 
20  years  ago  when  they  were  still  friends.  These  are 
far-out  alternativ.‘>s;  projections  may  serve  to  open 
possibilities.  I’m  sure  concerned  about  the  ones  that  end 
with  a  prediction.  One  thing  about  a  prediction;  You 
can  be  sure  it  will  be  wrong. 

Dr.  Slafkosky.  Let’s  talk  about  your  projections.  If 
you  are  interested  in  the  insight  you  theoretically  get, 
you  internally  are  doing  projections  and  you  may  get 
some.  It  »eems  fhe  value  of  these  insights  are  only  as 
good  as  your  projections.  It  is  only  afterwards  they 
receive  any  value.  It  you  agree  then  one  of  the  values 
in  this  procedure  is  looking  at  how  good  your  projec¬ 
tions  and  forecasts  were.  Only  by  this  reflectivity  can 
you  profit  internally  and,  as  matter  of  fact,  see  the 
implications  of  various  variables  or  interrelationships 
that  might  exist  in  the  area  you  are  discussing. 

Dr.  Johnston.  You  prompt  a  reply  that  makes  me 
sound  a  little  schizophrenic.  1  agreed  with  Dick  Irwin 
when  he  said  if  you  had  a  choice  between  a  wild  pro¬ 
jection  and  an  accurate  one,  he  would  work  on  the 
accurate  one.  This  schizophrenic  element  is  here.  The 
future  is  something  we  are  inventing.  It  is  a  function 
of  the  past  but  also  a  function  of  decisions  we  need  to 
make  and  the  question  is  what  do  we  want  to  be  and 
what  do  wc  want  to  do?  There  is  a  book  by  Dennis 
Gabor  and  I  quote  the  title:  “Inventing  the  Future,” 
which  can  sell  the  book.  I  read  it  and  I’m  not  as  satis- 

'  'ith  the  inside  as  I  am  with  the  title.  We  are 
involved  in  the  business  of  invention  and  not  just 
technological  invention.  Our  best  inventions  in  the  next 
50  yeKrs  are  going  to  be  in  the  social  area.  We  have 
seen  that  today  to  know  this  is  true. 
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Major  Martino.  How  does  an  internal  criterion  of 
infon^ation  measure  the  usefulness  of  a  forecast,  other 
than  how  accurate  it  was  as  viewed  years  afterwards? 
Another  problem — you  choose  what  you  consider  to  be 
the  most  likely  set  of  assumptions  and  then  make  a 
single  projection  which  becomes  a  forecast.  I  have  hud 
experience  with  problems  where  one  set  of  assumptions 
had  a  likelihood  of  10  percent.  This  was  the  most  likely 
set  of  assumptions.  But  there  were  others  quite  widely 
spaced  that  had  a  likelihood  cf  9  percent.  So  I  had  one 
most  likely  set  of  assumptions  but  it  wasn’t  much  more 
likely  than  something  radically  different.  The  point  is. 
I’m  changing  roles  for  a  moment  and  speaking  not  as 
a  forecaster  but  as  a  consumer  of  forecasts,  the  de¬ 
cisionmaker,  the  rdvisor  to  the  decisionmaker  who  has 
to  use  the  forecast.  The  statement,  this  is  the  single 
most  likely  set  of  assumptions,  is  not  nearly  as  useful 
as  telling  me:  Here  are  three,  or  whatever  the  number, 
about  equally  likely  things  that  could  happen.  Al,  how 
do  I  judge  the  utility  of  the  forecast?  1  judge  it  in 
terms  of  how  well  it  helps  me  plan  for  the  disasters 
that  might  happen,  and  do  this  within  my  budget.  It 
may  be  that  the  disasters-  never  happen.  If  I  have 
optimally  deployed  my  resources,  I  have  not  wasted 
them  on  an  unlikely  disaster.  But  I  have  deployed  them 
ill  defense  against  reasonably  likely  disaster;  then  if 
this  is  not  the  case  isnd  they  don’t  happen,  the  projec¬ 
tion  and  the  forecast  was  inaccurate  but,  nevertheless, 
it  was  useful  to  me. 

Dr.  Johnston.  I  think  you  made  the  point  better 
than  I  could.  Sometimes  you  make  a  projection  you 
hope  will  be  self-denying  and  sometimes  you  make  one 
you  hope  will  be  self-fulfilling.  The  government  in  its 
infinite  wisdom  is  always  doing  this.  When  a  projection 
about  a  labor  force  is  made  we  are,  in  eff-jct,  trving  to 
announce:  “Look,  folks,  we  are  going  to  have  a  hun¬ 
dred  and  five  million  people  that  are  going  to  be  work¬ 
ing  or  looking  for  work.  With  more  than  -5  percent  of 
them  looking  for  work,  we  will  have  troubles.  We’ve 
got  to  get  a  hundred  million  jobs."  We  hoiie  it  is  sclf- 
fulfilhug  in  the  sense  rhat  we  don’t  know  of  any  viable 
alternative  to  being  able  to  make  your  way  in  the 
world-of-work.  Someday  everybody  may  be  educate*! 
to  leisure  but  that  would  lie  an  altern.stive,  Tliis  self- 
denying  prophe<  y  is  also  useful  -the  one  vou  look  at 
10  years  later  and  say,  “Gee,  that  was  all  wet;  we 
didn’t  have  an  atomic-  war.  ’  It  was  useful  at  the  time 
because  it  led  to  policies  and  programs  which  made 
war  less  likely  and  caiisel  it  not  tc  hapf^en.  Certainly  a 
great  dca’  of  your  work  is  in  the  area — I  hope  all  of 
your  work  forever  will  he  self-denying  in  that  sense,  if 
you  prevent  what  everybody  hopes  won’t  happen. 


Mr.  Cetron.  I  was  glad  to  hear  you  mention  the  war 
and  the  ones  who  will  understand  the  work  in  the  future. 
Daddario,  in  our  Congress  now,  in  a  briefing  a  few 
weeks  ago  said  he  was  pushing  to  have  a  group  set  up 
reporting  directly  to  Congress,  indicating  what  the 
possible  future  world  will  look  like.  What  is  techno¬ 
logically  feasible,  socially  acceptable  and  the  second 
group,  not  this  same  in  Congress  saying,  “Now,  not 
1984  we  will  do  this  and  this,  but  how  do  we  puU  to¬ 
gether  and  coordinate  various  people  going  in  different 
directions  to  make  sure  the  better  goab  for  this  country 
will  be  achieved.’’  Part  of  the  rationality  brought  out 
was  Daddario  said,  “You  are  talking  about  1984;  you 
brought  it  hack  over  here;  you  are  going  to  direct  us; 
we’re  going  to  have  a  ruling  group — not  an  aristocracy 
— telling  us  what  the  future  will  be,”  and  I  think  he 
handled  it  well  by  saying,  “No,  I’m  not.  I’m  trying  to 
say  what  would  make  the  world  a  better  place  to  live  in, 
especially  the  United  States,  what  we  can  do  by  coordi¬ 
nating  the  technolog-/,  the  social  implications,  to  make 
sure  this  will  be  available  and,  how  is  this  going  to 
put  us  in  a  better  position  to  take  care  of  the  country 
as  a  whole.”  1  believe  this  is  the  point  we  arc  trying  to 
bring  out.  Your  output,  the  labor  force,  the  input  on 
the  census,  these  things  are  inputs  to  them  because 
they’ve  get  to  use  this  as  part  of  their  rationale.  The 
belter  your  figures  are,  oi  the  poorer,  take  your  choice, 
the  better  their  projections  will  be  with  the  social 
implications,  which,  is  where  you  start  your  projections 
of  lechnolcigical  forecasts.  There  is  a  major  tuming- 
around  on  the  outside,  a  ferris  wheel  in  a  sense,  and  in 
the  center  is  your  future  environment.  Things  are 
happening  and  we  are  trying  to  clear  the  focus  on  this 
crystal  ball.  We  are  on  target  but  every  year  we  adjust 
and  I  think  that’s  what  you  are  trying  to  do. 

Dr.  Johnston.  It  should  be  a  continuing  process. 

Mr.  Friedman.  Those  of  you  who  do  not  live  in  or 
around  Washington,  D.C.,  may  not  be  aware  the  local 
tabloids  devote  considerable  space  to  the  Federal  em- 
pIoy*  es  of  the  nation.  You  also  may  not  he  aware  of  a 
recent  article  which  suggested  we  extend  the  v:ork  day 
one-ha.  f  hour  by  extending  the  lunch  period  from  a  half 
hour  to  an  hour,  and  the  Fetieral  Government  would 
save  a  h'llion  dollars.  What  does  this  suggest  to  your 
projected  labor  force  figure?  How  does  this  affect  your 
thinking? 

Dr.  Johnston.  One  of  the  biggest  weaknesses  in  our 
labor  force  projections  is  we  treat  each  (lerson  as  a 
unit  and  a  person  could  be  working  for  pay  or  profit 
i  hour  a  week  or  more  and  he  is  in  the  labor  force. 
The  biggest  inknown,  and  this  mav  be  an  advantage, 
it  gives  some  leeway  in  planning  for  jobs  because  if 
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yon  are  getting  people  who  would  be  working  part  time 
and,  aome  won’t  work  under  any  other  conditions,  you 
may  talk  about  a  hundred  and  five  million  people  in 
the  labor  force,  but  in  terms  of  hours  of  paid  employ¬ 
ment,  that  would  have  to  be  provided — you  might  have 
a  plus  or  minus  factor  of  10  or  15  percent  depending 
on  the  willingness  or  readiness  of  the  people  to  work 
shorter  or  longer  hours.  We  do  need  to  refine  our  pro¬ 
jections  by  introducing  still  another  variable,  which 
has  to  be  projected,  namely,  an  sverage  of  hours 
worked.  We  think  of  two  thousand  hours  as  a  con¬ 
venient  work  year  pattern :  SO  weeks,  40  hours  a  week, 
give  or  take  a  we^  or  so.  But  it  Is  not  2,000  hours 
because  more  people  in  the  labor  force  are  coming 
down  to  1,800  and  for  others  it  is  i.alf  of  that  o'  a 
third.  We  need  to  have  that  variable  and  we  don’t. 

Mr.  Friedman.  If  the  purpose  was  to  extend  the  work 
day  a  half  hour,  eliminating  the  need  to  hire  ten  thou¬ 
sand  additional  people,  thereby  saving  a  billion  dollars. 
That  was  the  implication.  If  you  do  not  hire  ten  thou¬ 
sand  people  what  does  this  mean  to  your  unempioyn.ent 
force? 

Dr.  Johnston.  Much  of  our  work  is  at  cross  purposes 
because  what  you  are  saying  is:  You  are  going  to 
reduce  the  demand  and  demand  being  reduced  in  Gov¬ 
ernment  is  still  a  small  component  of  the  total  economy. 
Labor  is  a  very  expensive  component.  You  have  every¬ 
body  reducing  demand  then  what  are  we  going  to  do? 
One  possibility  is  this  leisure  world  that  David  Reisman 
has  been  talking  about.  I  daresay  it  is  going  to  take 
more  ediication  to  handle  leisure  than  it  ever  has  taken 
to  handle  work — education  at  knowing  ourselves  and 
what  we  are  like,  becoming  acquainted  with  ourselves. 
Most  of  us  can’t  find  out  about  ourselves  because  we 
are  too  busy  earning  a  living.  Maybe  that’s  good.  It 
looks  like  WPA,  disguised  under  different  initials.  The 
Government  has  been  suggested  as  an  emplover  of  last 
resort,  to  which  the  cynics  comment:  It  already  is 

Dr.  Chacko.  I  think  you  have  informatics  which 
would  take  you  out  of  that.  I  would  like  to  point  out 
two  things.  You  have  been  avoiding  your  responsi¬ 
bility— -not  you  personally — bv  saying  the  projections 
will  he  checked  against  numbers.  We  could  care  less 
about  that  and  should  he  more  responsible  to  our 
obligations  to  the  social  affairs.  If  you  were  to  say  what 
errors,  in  one  projection,  are  going  to  have  the  worst 
effect.  For  instance,  if  you  were  10  percent  off  in  pro¬ 
jecting  the  labor  force  in  the  age  group  .SO  to  60.  this 
would  be  leaS  of  a  calamity  than  the  same  kind  of  error 
in  predicting  the  labor  force  of  people  between  16  and 
24,  the  reason  being  the  older  age  group  have  the 
wherewithal  to  find  a  place  better  than  kids  just  enter¬ 


ing  the  labor  force  and  the  damage  that  would  be  done 
by  stretching  the  rubber  band  is  far  more  acute  in  the 
lower  age  level.  I  suggest  if  we  consciously  said,  “This 
is  one  position,  the  error  will  be  more  damaging  than 
something  else.”  This  is  what  you  are  doing  by  default, 
simply  saying  everyone  is  equal,  but  this  will  be  intro¬ 
ducing  a  design  having  a  format.  There  is  one  which 
could  work  for  projection  purposes,  which  is  the  com¬ 
pleted  families.  I  know  you  have  had  this^  analysis,  but 
I  don’t  know  what  you  are  doing  now.  If  a  round  figure 
of  three  children  is  taken  as  the  standard,  it  is  clear  the 
family  has  two  children  and  is  likely  to  have  one  more 
rather  than  two.  This  again  you  can  differentiate.  If  the 
woman  hed  two  children  before  the  age  of  25,  the 
probability  of  having  an  additional  child  is  higher 
than  if  the  woman  is  age  .'U)  when  she  has  two  children. 
We  can  narrow  the  area  with  this  information  and 
check  the  actual  data  of  family  completion  as  we  go 
along.  By  watching  this  vari.ible  we  can  improve  our 
projections.  You  spoke  of  the  econometric  projections 
and  later  you  talked  about  the  insensitivity  of  the  labor 
force  to  economic  factors  and  birth  rales.  The  only  way 
to  reconcile  the  two  statements  is  to  have  an  equation 
of  the  form: 

6,  b.,  6, 
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Dr.  Johnston.  Yes,  I  did  it  in  a  paper  for  a  course 
in  econometrics.  The  result  was  a  little  beyond  my 
capacitv  to  analyze  but  the  professor,  Martin  Abel,  a 
wonderful  teacher,  was  rather  disappointed.  Everything 
was  directly  contrary  to  what  we  would  be  able  to 
explain  on  the  basis  of  what  we  understand  about 
human  motivations.  Maybe  we  arc  wrong.  We  are 
approaching  the  results  of  sn  equation  with  notions  as 
to  how  people  respond.  Everything  increases  through  , 
time,  we  are  dealing  with  many  serial  correlations.  You/ 
get  beautiful  correlations  and  you  may  get  pretty  tigh^ 
coefficients  without  loo  much  variance.  .A  standard  de¬ 
viation  term  might  be  accepted  hut  you  rtre  left  with  an 
independent  variable,  which  you  have  to  make  assump¬ 
tions  about.  1  don’t  want  to  disagree  with  your  criti¬ 
cism,  that  you  don’t  mean  persorallv,  but  1  lake  them 
personally  because  we  are  an  underdeveloped  world. 
This  is  a  world  of  soft  science  technology  and  I  am  in 
a  position  whereby  f  ran  art  on  what  I  learn.  There 
are  only  four  of  us  working  on  this  and  we  can  do 
.something  about  is  thii  discriminate  analysis,  but  focus 
on  the  groups  that  are  important  and  you  are  exactly 
right.  From  16  to  24  years  is  a  problem.  Beyond  this 
age  people  have  settled  into  a  routine  or  have  the 
wherewithal  to  cope  The  young  entrants  art-  the  cru¬ 
cial  point.  I  am  making  an  educational  projection,  tied 


to  the  Census  Bureau’s  projections,  for  the  population 
as  a  whole.  Education  is  one  variable  that  is  subject  to 
policy  mani|>ulation  to  a  certain  extent  which  we  can 
introduce  at  a  faster  or  lower  rate.  1  don’t  know  about 
lowering  it,  but  we  can  increase  the  educational  growth 
to  an  extent.  You  are  right,  our  projection  is  deficient 
because  we  have  over-quantified.  A  person  in  the  labor 
force  who  is  going  to  work  40  years,  40  hours  a  week, 
52  weeks  a  year,  is  one;  and  a  person  drifting  in  and 
out  at  random,  working  a  few  hours  at  a  time  is  also 
one,  which  is  misleading. 

Major  Swett.  A  quick  billiard  with  Dr.  Slaf- 
’  .oXy’s  question  and  Major  Martino’s  response.  We  are 
addressing  the  problem  of  validity  of  forecasting  and 
what  we  have  is  this;  The  nature  of  all  forecasts  and 
projections  is  essentially  deductive.  The  sophistication 
of  technique  can  increase  the  validity  of  the  product  up 
to  but  not  beyond  that  of  its  entering  assumption.  The 
weakest  link  in  the  chain  of  social  sciences  is  the  enter¬ 
ing  assumption.  In  trying  to  establish  criteria  to  evalu¬ 
ate  forecasts,  we  find  assumptions  have  to  be  verified 
against  data.  Data  varies  by  credibility  with  time  and 
we  have  none  for  the  future.  This  is  one  reason  you 
can’t  check  a  forecast  against  the  future.  The  other  is 
the  feedback  effect.  If  anybody  knows  about  your  fore¬ 
casts  to  th.'t  degree  it  is  a  steering  signal  that  changes 
the  future  condition  and  has  a  self-defeating  or  self- 
fulfilling  aspect.  The  future  is  no  good  to  evaluate 
forreasts,  even  when  you  get  there.  As  far  as  checking 
against  present  data  is  concerned,  you  don’t  have  it 
and  it  is  not  instantly  available.  There  is  an  optimum 
point  in  the  past  but  you  can’t  go  back  for  data  to 
verify  your  assumptions  because  data  becomes  expo¬ 
nentially  uncertain  as  you  go  backwards.  The  farther 
you  go  back  the  less  is  known  about  specifics  to  check 
your  theory  against.  There  is  an  optimum  pong  in  the 
not  distant  past  where  data  is  most  valid  for  the  veri¬ 
fication  of  assumptions  or  premises.  In  the  area  of 
international  relations,  which  is  mv  field,  data  is  best 
at  .5  to  8  years  past.  The  criterion  we  are  using  to 
evaluate  forecasts  is  the  degree  to  which  entering  as¬ 
sumptions  have  lieen  empirically  verified.  The  interest¬ 
ing  thing  is  that  some  basic  assumptions  in  my  disci¬ 
pline  have  lieen  ep.jiirically  denied,  hut  continue  to  be 
used  as  input  to  scenario  generators  and  forecasting 


systems.  The  way  out  would  be  to  increase  the  efforts 
to  verify  or  deny  the  assumptions  on  which  'hesi;  pro¬ 
jections  are  based. 

Dr.  Johnston.  We  think  of  this  as  the  ego  problem. 
Yoj  have  to  reach  in  and  out. 

Mr.  Hacke.  The  reported  unemployment  rate  under¬ 
estimates  actual  unemployment  and  is  self-limiting  be¬ 
cause — particularly  to  certain  structural  elements  of 
population — no  one  can  receive  jobs  after  a  certain 
point.  Would  you  comment  on  this? 

Dr.  Johnston  Yes,  the  claim  is  valid.  You  can  al¬ 
ways  suggest  alternative  measures  but  the  measure  we 
use  on  unemployment  makes  it  impossible  for  this  to 
be  valid.  You  were  talking  in  terms  of  the  real  world 
and  we  are  talking  in  terms  of  how  we  make  a  measure. 
We  measure  unemployment  by  surveying  a  sample  of 
households  about  one  out  of  every  1,100  households  a 
month.  We  cover  around  52,000  households,  the  best 
sample  of  any  survey  in  this  country — and  f  want  to 
compliment  my  census  colleagues  on  the  wonderful 
work  they  do  in  that  area.  They  do  a  survey  each 
month.  We  measure  unemployment  by  responses  to 
questions:  What  were  you  doing  lest  week’  Was  he 
looking  for  work,  or  unemployed?  We  miss  the  people 
we  undercounted  initially,  maybe  5  million  people.  Are 
those  people  more  likely  to  be  unemployed?  We  can’t 
jump  to  that  conclusion.  It  has  been  tried  by  or.e  con¬ 
ference.  Your  question  addresses  itself  more  sensitively 
and  our  Secretary  of  Labor  is  putting  it  in  these  terms: 
How  many  of  our  people  are  underemployed?  You  can 
he  underemployed  if  you  are  not  using  your  education. 
Then  you  can  easily  slip  from  that  to  the  psychologist’s 
point;  We  are  underemployed  if  we  are  not  using  our¬ 
selves  to  the  fullest  capacity.  I,  personally,  don’t  like 
to  l>e  utilized  to  the  fullest  capacity  because  it  makes  me 
feel  tired  all  the  time.  There  is  such  a  thing  as  over¬ 
employment  and  1  am  overemployed  but  we  don’t  know 
how  to  measure  it.  Some  people  are  discouraged.  In  the 
.survey  of  .50,000,  under  age  65,  we  found  a  small  num¬ 
ber  of  these.  Many  [>eople  are  discouraged  because 
thev  are  getting  one  day  older  every  day.  This  is  dis¬ 
couraging  for  all  of  us.  We  can  attribute  this  discour¬ 
agement  to  something  else,  hut  it  is  di^ouraging  to 
get  older  and  we  don’t  know  how  to  measure  this  very 
well. 


Mr.  Hacke  describes  a  study  he  has  recently  complcterl,  that  requirerl  him  to  in¬ 
vestigate  several  forecasting  techniques  which  also  involves!  trying  to  forecast  "what 
might  have  been  if  history  had  been  different.  His  cautionary  words  about  how 
difficult  it  is  to  find  out  what  artuaily  hap|>ened  in  the  pa.sl,  let  alone  make  forecasts 
for  the  future,  should  l>e  taken  to  heart  by  all  forecas*'‘rs. 

— Editor 
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ANTICIPATING  SOCIOECONOMIC  CONSEQUENCES  OF 
A  MAJOR  TECHNOLOGICAL  INNOVATION 

’  f 

James  f.  Hacks,  Jr.,  Stanford  Rosoareh  Institute 


INTRODUCTION 


This  is  a  report  on  two  related  research  projects. 
One  was  to  produce  a  manual  on  technological  fore¬ 
casting  methods  for  the  long  Range  Planning  Service 
(LRPS)  of  Stanford  Research  Institute.  This  project 
has  produced  two  volumes,  now  in  galley  proofs,  en¬ 
titled  “Forecasting  Technological  Change:  Exe-mtive 
Overview  and  Forecasting  Technological  Change:  Man¬ 
ual  for  Analysts.” 

The  other  project  was  for  the  National  Science  Foun¬ 
dation  to  assess  the  feasibility  of  anticipating  economic 
and  social  consequences  of  a  major  technological  inno¬ 
vation.  A  paper  that  I  gave  last  year  at  this  symposium 
embodied  preliminary  results  of  this  study.  Its  title  was 
“A  Methodological  Preface  to  Technological  Forecast¬ 
ing”,  and  it  sought  to  prove  the  thesis  that  technological 
forecasting  has  the  seme  epistemological  warrants  as 
such  other  applications  of  scientific  knowledge  as  engi¬ 
neering  and  medicine. 

The  final  report  on  this  NSF  project  has  just  been 


released  for  publication.^  In  broad  outline,  the  research 
plan  was  to  assemble,  analyze,  evaluate,  and  organize 
existing  methods  for  forecasting  technological  .change; 
to  assemble,  analyze,  evaluate,  and  organiz^ existing 
methods  for  assessing  socioeconomic  impa^  of  past 
technological  innovations;  to  combine  the  two  sets  of 
methods  into  methods  for  anticipating  socioeconomic 
impacts  of  future  technological  innovations;  and  to  try 
the  methods  out  on  a  past  innovation  to  see  how  they 
work.  * 

This  paper  therefore  falls  naturally  into  four  main 
perts,  following  this  introduction:  Technological  fore¬ 
casting,  assessing  socioeconomic  impacts,  testing  the 

combined  methods-  and  conclusions.  First,  methods  of 

forecasting  technological  change.  ! 

‘James  E.  Hacke,  Jr.,  “The  Feasibility  of  Anticipating  Eco¬ 
nomic  and  Social  Consequences  of  a  Major  Technological  Inno¬ 
vation”  (SRI,  May  1967).  { 

‘  V 


TECHNOLOGICAL  FORECASTING 


In  his  monumental  volume,^  Erich  Jantsch  described 
more  than  1(X)  techniques  that  he  calls  methods  of 
technological  forecasting.  But,  in  the  first  place,  he 
includes  all  human  activity,  from  basic  science  to  gov¬ 
ernment,  in  the  subject  matter  of  bis  forecasting;  sec¬ 
ond,  he  includes  diffusion  (among  disciplines,  indus¬ 
tries, .  nations.  and  cultures),  along  with  innovation, 
as  something  to  forecast;  and,  third.,  he  advocates  what 
he  calls  normative  forecasting,  or  goal  formulation  and 
planning,  among  the  forecasting  tasks. 

Both  for  methodological  and  for  practical  reasons^ 
the  scope  of  the  forecasting  task  here  studied  is  much 
more  limited.  We  differentiate  technology  from  science,  ^ 
economics,  sociology,  and  politics;  study  innovation 

‘“Technological  Forecasting  in  Perspective”  (OECD,  Paris. 
1967). 


and  not  diffusion  and  leave  goal  formulation  and 
planning  to  others. 

Technological  change  is  not  a  process  that  goes  li!:e 
clockwork,  with  every  tick  highly  predictable  from  the 
sequence  of  past  ticks.  Individual  technological  ad¬ 
vances  are  almost  completely  unpredictable.  So,  if  you 
will,  technological  change  is  like  hunian  birth.  Each 
birth  is  the  result  of  fbighjy  unpredictable  activities; 
but  the  birth  rate  is^ot  completely  m:predictable. 

We  cannot  for^^st  technological  change  with  com¬ 
plete  confidence  accuracy,  any  more  than  we  car. 
forecast  the  hiiUk  ra^'  with  complete  confidence  and 
accuracy.  At  mtjpt/w  'pm  forecast  that  technological 
change  will  probatof  fall  within  a  certain  range.  The 
real  question  is  whbpef  such  “iffy”  forecasts  are  useful 

Technological, change  does  not  go  on  in  a  vacuum.  It 
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is  largely  the  result  of  technological  effort,  and  that 
efforr  is  motivated  largely  by  economic  and  social 
concerns.  Technological  innovations  in  turn  have  eco¬ 
nomic  and  social  consequences:  think  of  the  Bomb,  the 
Pill,  and  LSD.  In  my  opinion,  nevertheless,  technologi¬ 
cal  development  is  much  more  dependent  on  socio¬ 
economic  developments  than  vice  versa.  This  is  a  basic 
contention  underlying  the  whole  study. 

In  the  light  of  this  contention,  the  rational  approach 
to  technological  forecasting  is  to  start  with  forecasts 
of  economic  and  social  conditions,  goals,  values,  be¬ 
liefs,  and  concerns.  These  forecasts  can  assist  in  judging 
-whether  specific  technological  efforts  are  likely  to  re¬ 
ceive  more,  less,  or  the  same  support  in  the  future  as 
in  the  past.  These  estimates  are  the  principal  bases  for 
estimating  whether  future  technological  trends  are 
likely  to  follow  historical  patterns  or  to  diverge  up  or 
down. 

Having  made  a  technological  forecast,  the  analyst 
should  investigate  possible  economic  and  social  con¬ 
sequences  of  the  forecast  technological  chioges.  If 

QUALITATIVE 

The  five  qualitative  methods  are  those  that  forecast 
the  probable  pattern  of  future  development.  These 
methods  can  give  insight  into  such  aspects  of  the  future 
as  the  layout  and  organization  of  the  metropolitan  area; 
the  nature  of  future  transportation  and  communication 
systems;  and  relations  between  men  and  machines  in 
the  future  business  organization. 

The  first  of  these  five  qualitative  methods  is  delineat¬ 
ing  the  boundaries  of  the  possible.  These  boundaries  are 
set  by  the  definition  of  terms,  budgetary  identities,  the 
laws  of  science,  the  present  technological  situation,  and 
the  nature  of  human  utility.  Budgetary  identities  ap¬ 
ply  not  only  to  money  but  to  manpower  and  material 
resources.  Any  future  development  of  the  U.S.  railroad 
system  must  start  with  rails  that  are  now  4  feet,  8V2 
inches  apart.  Telephony  between  the  United  States  and 
India  will  be  of  limited  utility  as  long  as  they  sleep 
while  we  work  and  vice  versa. 

The  second  qualitative  method  is  historical  analogy. 

We  uSe  it  whenever  we  use  metaphorical  labels  like 
“iron  horse”  and  “industrial  revolution”.  But  the  only 
systematic  attempt  to  apply  historical  analogy  to  tech¬ 
nological  forecasting  of  which  I  am  aware  is  In  a  book 
edited  by  Bruce  Mazlich.® 

By  comparison  and  contrast  with  another  great  social 
invention,  the  transcontinental  railroads,  Mazlich  and 

*“The  Railroad  and  the  Space  Program:  An  Exploration  in 
Historical  Analogy”  (MIT  Press,  1965). 


necessary,  he  should  modify  the  socioeconomic  fore¬ 
casts  that  he  started  with,  and  repeat  the  process  until 
the  socioeconomic  and  the  technological  forecast  are 
consistent,  y 

The  analysis,  evaluation,  and  classification  of  fore¬ 
casting  methods  in  this  study  resulted  in  10  generic 
classes  of  forecasting  methods,  five  qualitative  and  five 
quantitative.  The  qualitative  methods  yield  a  picture  of 
the  probable  fiiture  pattern  of  technological  develop¬ 
ment.  Quantitative  methods  yield  the  prr-hable  future 
change  in  parameters  descriptive  of  the  state  of  the  art 
in  a  technological  discipline.  A  parameter  is  meaningful 
only  in  the  context  of  a  known  technological  pattern; 
and,  when  a  parameter  reaches  a  critical  value,  a 
change  in  pattern  may  become  possible  or  necessary. 
So  you  can  start  either  with  patterns  or  with  param¬ 
eters.  Either  way,  you  must  continue  investigating  the 
probable  effects  of  each  on  the  other  until  you  have  a 
forecast  that  is  qualitatively  and  quantitatively  con¬ 
sistent. 

s 

METHODS  'A 

his  authors  try  to  draw  conclusions  about  the  space 
effort  and  about  the  general  nature  of  social  invention 
and  technological  effort.  One  of  their  conclusions: 
Innovations  do  not  move  from  one  industry  to  another 
primarily  through  literature,  but  in  th6  minds  of  trained 
employees. 

The  third  kind  of  qualitative  forecasting  method  is 
perhaps  the  most  widely  used :  expert  opinion.  At  first, 
it  seems  reasonable  to  go  to  an  electronics  expert  and 
say,  “Give  me  a  forecast  of  the  future  of  electronics”; 
but  this  procedure  is  not  so  v;ise  as  it  seems.  First,  he 
is  not  necessarily  an  expert  in  forecasting.  He'  will 
almost  certainly  invent  his  own  forecasting  methods, 
without  being  aware  of  what  has  been  learned  about  it. 
In  the  second  place,  he  is  likely  to  be  ignorant  of 
developments  in  other  fields  that  are  going  to  have  a 
profound  effect  on  his  own.  In  1957,  when  I  forecast 
the  future  of  global  telecommunication,  I  did  not  take 
space  communication  into  account. 

It  is  therefore  usually  be.  er  to  rely  on  the  technical 
expert  only  for  the  present  conditions  in  his  discipline 
— including  the  present  rate  of  development  and  the 
nature  and  severity  of  presently  visible  barriers  to 
future  development.  On  the  basis  is  these  data,  the  fore¬ 
caster  can  erect  a  projection  of  probable  future  develop¬ 
ments.  He  will  want  to  check  this  forecast  with  the 
expert,  to  assure  that  he  has  made  no  technical  blunder; 
but  the  responsibility  for  it  is  his. 
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By  contrast,  the  fourth  qualitative  forecasting 
method,  using  consensus,  makes  deliberate  use  of  the 
intuitive  forecasts  of  a  variety  of  experts.  The  assump¬ 
tion  here  is  that  the  experts  can  bring  a  lot  of  knowl¬ 
edge,  rem'”nbered  and  unremembered,  to  bear  on  the 
forecasting  task.  If  the  group  includes  a  wide  variety 
of  disciplines  and  economic  and  social  viewpoints,  they 
can  help  correct  for  one  another’s  prejudices  and 
blindnesses. 

Many  ways  of  arriving  at  a  consensus  are  open  to 
the  objection  that  the  experts  can  influence  one  another 
psychologically  as  well  as  logically.  The  Delphi  tech¬ 
nique  developed  by  Olaf  Helmer  and  others  at  Rand 
helps  to  obviate  this  limitation. 

Consensus  does  not  necessarily  mean  unanimity.  The 
experts  may  agree  to  diss'-ree.  This  spread  of  expert 
opinion  can  serve  as  an  ertainty  measure  for  the 
forecast. 

These  first  four  qualitative  forecasting  methods  are 
useful  as  far  as  we  can  reason  in?-  the  future  from  the 
present  and  the  past.  This  time  span  is  adequate  for 
almost  all  planning  activities.  But  on  some  occasions  it 
would  be  advantageous  to  have  some  idea  of  the  future 


as  far  as  50  years  hence.  Young  people  in  school  today, 
for  instance,  may  still  be  working  in  A.D.  2020.  Edu¬ 
cators  would  appreciate  an  inkling  of  what  skills  known 
today  will  st'II  be  useful  then. 

The  last  method  of  qualitative  forecasting  is  useful 
for  discerning  what  is  discernible  about  such  a  remote 
future.  It  has  been  called  inventing  alternatWe  futures, 
and  involves  writing  as  many  different,  plausible,  in¬ 
ternally  consistent  future  histories  as  possible:  of  the 
world,  of  this  Nation,  and  of  relevant  technology.  From 
these  alternatives  emerges  an  envelope  enclosing  the 
long-range  technological  changes  that  may  be  expected 
to  occur. 

As  a  matter  of  experience,  only  about  a  score  of 
rei.ily  different  alternative  world  futures  have  so  far 
been  invented.  That  there  are  so  few  may  reflect  the 
limits  of  human  imagination  rather  than  of  the  possible. 
But  they  surely  embrace  more  alternatives  than  a  less 
extensive  and  systematic  effort  would  produce,  and 
thus  yield  the  best  available  basis  for  preliminary  long- 
range  planning.  And  there  will  be  plenty  of  time  to 
change  plans  to  deal  with  the  unexpected. 


QUANTITATIVE  METHODS 


Before  discussing  quantitative  forecasting  methods  in 
detail,  consider  for  a  minute  how  to  select  and  collect 
data  on  quantities  to  forecast. 

A  quantitative  measure  of  a  technology  should  indi¬ 
cate  something  about  how  well  we  are  likely  to  be  doing 
something  tomorrow,  as  compared  with  today.  The 
chosen  quantity  should  therefore  actually  represent  the 
technology  in  question.  Speed  records,  for  instance,  do 
not  represent  how  fast  our  transportation  systems  ac¬ 
tually  move  goods  and  people. 

Unfortunately,  most  technological  data  were  not 
gathered  for  forecasting,  but  for  accounting  and  con¬ 
trol.  They  are  not  always  particularly  appropriate  as  a 
base  for  forecasting. 

Early  in  the  development  of  a  technology,  there  is 
considerable  uncertainty  about  what  data  are  suitable 
for  any  purpose,  and  there  is  little  or  no  systematic 
effort  at  collecting  data.  The  forecaster  must  then  be 
satisfied  with  very  approximate  methods,  until  system¬ 
atically  collected  and  organized  data  become  available. 

Quantitative  forecasting  methods  also  fall  into  five 
generic  categories. 

When  you  have  collected  and  organized  your  data, 
you  can  hardly  help  plotting  them  as  a  function  of  time 
and  '.eeing  what  the  trend  looks  like.  Forecasting  by 
extending  past  trends  into  the  future  is  the  first  quan- 


titativ  method:  inductive  trend  determination  and 
extrapolation.  It  is  inductive  because  it  requires  no 
explanation  of  why  the  trend  is  as  it  is.  It  just  is,  and 
that  is  enough  for  the  forecaster. 

In  1957,  James  E.  Rosenzweig  *  plotted  aluminum 
consumption  data  for  the  United  States  for  the  years 
1919-54  inclusive  (see  fig.  VlI-1) .  He  extrapolated  the 
observed  trend  to  a  forecast  demand  for  8.59  billion 
pounds  in  1965. 

Deviations  from  a  straight-line  course  are  probably 
easier  to  see  then  deviations  from  anything  else,  so 
the  analyst  is  happiest  if  he  can  manipulate  his  data 
to  show  a  straight  line  trend.  Rosenzweig  used  a  loga¬ 
rithmic  scale  for  aluminum  demand.  Other  transf onna- 
tions  are  possible. 

Then  trend  line  can  be  dravm  and  extended  by  eye; 
but  almost  always  the  analyst  will  use  statistical 
methods.  There  are  two  pitfalls  to  watch  for  in  such 
use  of  statistical  method. 

The  first  is  to  take  the  trend  line  as  the  forecast, 
without  realizing  that  future  values  may  vary  from  the 
trend  by  as  much  as  past  values  have.  A  main  reason 
for  my  using  Rosenzweig’s  aluminum  forecast  as  an 

'In  his  “The  Demand  for  Aluminum:  A  Case  Study  in  Long- 
Range  Forecasting”  (University  of  Illinois  “Bulletin,”  54,  63, 
April  1957). 
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Figure  VII-L-  Caloulsted  alur'inuir  a  as-  mption  in  the  United  States,  1910-65. 


example  is  his  calculating  and  supplying  the  standard 
error  of  his  forecast.  His  forecast  of  demand  for 
aluminum  in  1965  was  one-tenth  of  a  standard  devia¬ 
tion  off  from  the  actual  demand. 

But  the  second  pitfall  to  avoid  is  to  take  statistical 


mc-iures  of  variance  to('  seriously.  These  measures 
were  derived  on  t^e  assumption  tbit  ueviations  from 
the  tr  jnd  are  independent  of  one  another ;  and  hialorical 
time  series  rarely  show  random  deviations.  Rosen- 
zweig’s  aluminum  data  show  a  coefficient  of  serial  ccr- 
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relation  of  -(-0.80.  Each  point  is  like  a  moving  average 
over  a  period  significantly  longer  than  a  year,  and  so 
the  actual  fluctuation  in  demand  for  aluminum  is  much 
greater  than  the  data  reveal. 

So  don’t  take  a  trend  line  as  representing  what  is 
going  to  happen,  without  allowing  for  variance;  and 
don’t  take  statistical  measures  of  variance  too  seriously 
if  serial  correlation  is  present.  Taken  together,  these 
considerations  may  make  a  good-looking  trend  line 
rather  useless.  A  95-percent  confidence  interval  thai 
covers  a  100-to-l  range  may  be  of  limited  utility. 

The  second  kind  of  quantitative  forecast  is  even  more 
subject  to  pitfalls  than  inductive  tiend  determination 
and  extrapolation.  It  is  the  attempt  to  find  and  display 
periodic  fluctuations  in  the  value  of  a  parameter. 

In  two  circumstances,  searching  for  periodicity 
makes  sense.  One  is  when  some  reflexive  mechanism 
behind  the  phenomenon  under  study  makes  it  fluctuate 
periodically.  For  instance,  some  diseases  may  at  one 
time  have  traveled  around  the  world,  returning  to  the 
site  of  a  foimer  epidemic  when  the  population  there 
had  lost  most  of  its  immunity.  This  may  have  caused 
periodically  recurring  epidemics.  But  technological 
change  has  largely  destroyed  this  reflexive  mechanism, 
if  it  ever  existed.  Diseases,  as  well  as  people,  now 
travel  faster;  but  immunization  and  antibiotics  offer 
new  controls  of  epidemics. 

The  other  circumstance  warranting  search  for  period¬ 
icity  is  control  by  a  naturally  recurring  phenomenon, 
such  as  time  of  day  and  time  of  year.  For  example,  the 
average  daily  production  of  electricity  in  the  United 
States  shows  a  regular  secular  trend;  but  it  also  shows 
a  5-percent  semiannual  fluctuation  about  that  trend, 
peaking  in  summer  and  in  winter  (  see  fig.  VII-2  and 
VII-3). 

Cyclic  analysis  has  become  highly  sophisticated,  for 
example,  in  analysis  of  communication  traffic  demand. 
A  somewhat  different  technique,  harmonic  analysis,  has 
been  borrowed  from  communication  theory  by  C.  W.  J. 
Granger  to  forecast  the  sunspot  cycle.'  But  in  all  of 
these  instances  the  physical  basis  of  the  periodic  fluc¬ 
tuation  is  evident.  Human  beings  are  so  prone  to  see 
cycles  where  none  exist,  and  have  wasted  so  much  time 
trying  to  forecast  the  stock  market  from  sunspot  cycles, 
that  the  warning  is  probably  justified;  Don’t  use 
periodic  analysis  unless  you  know  how  the  periodicity 
comes  about  and  why. 

These  first  two  quantitative  methods  are  largely  in¬ 
ductive  and  heuristic.  By  contrast,  the  third  is  deduc- 

’In  "Spectral  Analysis  of  Economic  Time  Series”  (Princeton 
University  Press,  1960) . 


tive  and  theoretical.  It  consists  in  applying  to  the  data 
growth  curves  derived  from  theoretical  assumptions 
about  the  mechanism  by  which  the  parameters  change. 
In  a  pap^r  at  the  Lake  Placid  Club  last  spring,  for 
example,  Italph  Lenz  applied  to  military  aircraft  speeds 
a  curve  derived  under  the  assumption  that  the  speeds 
of  which  aircraft  are  capable  change  annually  by  a 
factor  proportional  to  currently  achieved  speed,  times 
a  factor  that  goes  to  zero  as  achieved  speed  approaches 
orbital  speed  (see  fig.  VII— 4.) 

Forecasting  by  use  of  theoretically  derived  laws  of 
change  is  subject  to  the  same  statistical  limitations  as 
trend  extrapolation.  Slight  changes  in  the  assumed 
mechanism  of  change  can  yield  large  differences  in  the 
forecast.  But  testing  a  variety  of  plausible  assumptions 
against  the  data  may  yield  some  insight  into  what  is 
going  on  to  cause  the  observed  change. 

The  three  preceding  methods  seek  to  express  the  ob¬ 
served  and  predicted  changes  directlv  as  a  function  of 
time.  The  fourth  quantitative  method,  trend  correla¬ 
tion,  expresses  the  measured  quantity  as  a  function  of 
other,  intervening  variables,  themselves  considered 
functions  of  time. 

At  first,  this  might  seem  like  exchanging  one  fore¬ 
casting  task  for  many,  with  no  clear  advantage. 
But  if  the  intervening  variables  chosen  are  actually 
important  influences  on  the  parameter  under  study  (or 
conversely),  the  residual  variance  can  be  significantly 
reduced.  After  his  inductive  trend  determination  and 
extrapolation  of  aluminum  demand,  Rosenzweig  inves¬ 
tigated  the  effect  of  several  intervening  variables  and 
chose  two  as  significant:  GNP  and  aluminum/steel 
price  ratio.  A  multiple  correlation  with  these  two  inter¬ 
vening  variables  reduced  his  standard  error  of  esti¬ 
mate,  for  1965,  to  20  percent. 

The  same  caveats  about  variance  and  serial  correla¬ 
tion  apply  to  multiple  correlation  as  to  those  methods 
that  study  phenomena  directly  as  functions  of  time.  In 
addition,  the  form  of  the  relationships  among  the  de¬ 
pendent  and  the  intervening  variables  is  uncertain  and 
is  not  often  investigated  in  any  great  depth. 

The  final  method  of  quantitative  forecasting  consists 
in  making  a  complete  model  of  the  process  under  study. 
The  model  may  be  analytical,  digital,  or  stochastic. 
\ny  attempt  to  be  rigorous  and  precise  in  such  a  model 
leads  to  many  equations  and  many  variables,  and  there¬ 
fore  is  best  treated  with  a  computer. 

The  demographic  projection  methods  described  in 
a  preceding  paper  by  Richard  Irwin  were  complete 
modeling  techniques,  of  the  particular  variety  that 
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uses  difference  equations.  As  his  discussion  illustrated,  inevitably  back  from  considering  quantitative  to  con- 
the  attempt  to  erect  a  complete  model  leads  you  almost  sidering  qualitative  changes. 


IMPAaS  OF  INNOVATIONS 


So  much  on  methods  of  technological  forecasting. 
More  detail,  and  more  justification  of  some  of  my 
statements,  is  available  in  my  NSF  and  LRPS  reports. 

As  you  remember,  the  basic  idea  was  to  combine 
methods  of  technological  forecasting  with  methods  of 
assessing  economic  and  social  impacts  of  past  innova¬ 
tions.  I  thought  that  assessing  past  innovations  had 
been  well  studied,  so  I  tackled  forecasting  first.  Then 
I  turned  to  methods  of  assessing  socioeconomic  impact. 

To  my  surprise,  all  of  the  studies  I  could  find  were 
guilty  of  the  post  hoc,  ergo  propter  hoc  fallacy.  If  they 
were  studies  of  hybrid  corn,  they  blamed  all  the 
changes  in  American  agriculture  on  hybrid  corn.  If 
they  were  studies  of  farm  mechanization,  they  blamed 
all  the  changes  on  the  tractor.  And  so  on.  I  got  an 
economic  historian  to  make  a  more  exhaustive  literature 
search  anu  analysis  for  me,  and  he  came  to  the  same 
conclusion. 

In  this  impasse,  I  arrived  at  three  propositions: 

1.  There  are  no  technological  forecasting  methods. 


There  are  just  forecasting  methods.  Most  of  the  meth¬ 
ods  I  have  outlined  were  in  fact  developed  first  for 
biological,  economic,  or  social  forecasting.  The  method 
appropriate  in  a  given  set  of  circumstances  does  not 
depend  so  much  on  whether  it  is  technological,  eco¬ 
nomic,  social,  or  demographic  phenomena  so  much  as 
on  the  morphology  of  the  data. 

2.  A  method  of  assessing  socioeconomic  conse¬ 
quences  of  an  innovation  that  escapes  at  least  partly 
from  the  post  hoc,  ergo  propter  hoc  fallacy  is  as 
follows : 

Describe  relevant  developments  following  introduc¬ 
tion  of  the  innovation;  describe  relevant  following  de¬ 
velopments  were  the  innovation  not  introduced;  and 
take  the  difference  between  the  two  sets  of  develop¬ 
ments.  This  is  the  impact  of  the  innovation. 

3.  It  makes  little  difference  whether  it  is  past  or 
future  impact  you  are  studying.  When  it  is  past,  you 
have  one  extrapolation  into  the  unknown  to  make: 
What  would  have  happened  had  the  innovation  not 
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been  introduced?  When  the  impact  is  in  the  future, 
you  have  two:  What  will  happen  if  the  innovation  is 
introduced,  an  1  what  will  happen  if  it  is  not? 

This,  then,  -  .  kutihod  developed  for  anticipating 

socioeconomic  consequences  of  a  major  technological 
innovation: 

Using  the  forecasting  methods  analyzed  and  organ¬ 
ized,  describe  what  might  happen,  or  might  have  hap¬ 
pened,  were  the  innovation  not  introduced;  describe 
what  happened  or  might  happen  after  its  introduction; 
take  the  difference.  The  remaining  task  was  to  test  the 
method,  by  applying  it  to  a  post  innovation. 

Example:  The  Transistor. — After  consultation  with 
the  National  Science  Foundation  and  a  fair  amount  of 
analysis  as  to  what  would  constitute  a  suitable  innova¬ 
tion  for  study,  we  chose  the  transistor.  The  transistor 
could  be  fairly  unambiguously  defined;  I  could  under¬ 
stand  it;  and  we  felt  that  plenty  of  data  would  be  avail¬ 
able.  At  the  same  time,  it  was  not  so  minor  as  to  con¬ 
stitute  no  test  of  the  method. 

The  idea  was  to  go  back  in  imagination  to  some  point 
in  past  time  and  use  only  data  available  at  that  time, 
and  see  if  we  could  make  a  forecast  using  that  data 
extending  to  some  point  in  the  more  recent  past.  Com¬ 
parison  of  the  forecast  with  actuality  would  constitute 
the  test  of  the  method. 

Our  first  intention  was  to  cut  off  at  the  end  of  1956, 
thus  allowing  a  forecast  over  the  decade  1956-66.  But 
the  available  information  through  1956,  both  qualita¬ 
tive  and  quantitative,  was  just  not  extensive  enough 
for  meaningful  forecasting.  So  we  moved  the  cutoff  to 
the  end  of  1957.  This  gave  us  two  9-year  periods:  from 
the  announcement  of  the  transistor  in  December  1948 
through  1957 ;  and  1958-66  inclusive. 

I  began  by  describing  economic  and  social  conditions 
in  1957,  giving  evidence  that  would  ju  tify  a  fore¬ 
caster’s  thinking  that  major  social  and  economic  trends 
would  contiiiiiS  for  a  decade  as  they  had  in  the  recent 
past.  Among  this  evidence  was  a  forecast  made  for 
Tempo  in  1957.* 

Between  1941  and  1957,  the  electronics  industry  had 
grown  about  11  perceni  per  year;  but  there  were  signs 
of  stagnation  or  maturity  in  1957.  Production  increase 
between  1956  and  1957  was  only  4.6  percent,  and  many 
indexes  were  off.  But  three  technological  developments 
often  associated  with  the  transistor  were  already  in 
being,  and  had  arisen  independent  of  the  tran.ustor: 
miniaturization,  printed  circuits,  and  computers. 

Historical  quantitative  data  on  the  early  history  of 
the  transistor  proved  unexpectedly  difficult  to  come  by. 

*“1970:  Challenge  to  Planners”  (General  Electric  Co.,  Santa 
Barbara). 


JETEC — ^later  EIA — had  accumulated  and  circulated  a 
lot  of  data,  but  they  were  proprietary  and  therefore  not 
available  for  this  study.  It  proved  necessary  to  tabulate 
what  data  we  could  from  the  trade  press  and  from 
advertisements. 

In  this  way  we  were  able  to  develop  four  time  series 
that  gave  some  basis  for  extrapolation :  Total  transistor 
production  (fig.  VII— 5)  transistor/tube  price  ratio, 
number  of  transistor  types,  and  performance  parameter 
(fig.  VII-6).  This  parameter,  pdfc®,  where  pa  is  device 
power  dissipation  and  fc  is  maximum  operating  fre¬ 
quency,  is  a  quite  generally  appropriate  figure  of  merit 
for  active  electronic  devices  originally  developed  for 
microwave  tubes.  In/  addition,  we  had  some  data  on 
foreign  transistor  production,  but  not  enough  for 
meaningful  time-series  analysis. 

Analysis  of  and  extrapolation  from  these  data  led  to 
the  following  quantitative  forecast  of  the  state  of  the 
transistor  art: 

By  1965,  U.S.  transistor  production  will  have  reached  at  least 
6  billion  units,  and  may  possibly  reach  as  many  as  600  billion. 
They  will  have  become  thoroughly  competitive  in  price  with 
vacuum  tubes:  the  ratio  of  transistor  to  tube  prices  will  average 
somewhere  between  a  third  and  three  times  as  much  per  unit. 
In  these  circumstances,  performance  characteristics  will  be  fully 
as  important  as  price  in  dictating  which  device  is  chosen.  Tran¬ 
sistors  with  a  dissipation  rating  of  1  watt  will  have  cutoff  fre¬ 
quencies  as  high  as  50  Gc/s.  There  will  be  at  least  2,650  regis¬ 
tered  transistor  types,  and  the  number  may  possibly  run  into 
the  hundreds  of  thousands,  although  it  is  difficult  to  see  how  so 
many  types  could  be  registered,  kept  track  of,  and  advantage¬ 
ously  used.  Other  free  world  countries  will  compete  effectively 
with  the  United  States  in  producing  and  marketing  transistors. 
Of  these,  Japan,  already  producing  a  fifth  as  many  as  the  United 
States,  is  in  a  particularly  strong  posiion. 

Two  of  the  quantitative  forecasts  proved  greatly  over- 
optimistic.  Transistor  production  in  1965  was  only 
one-seventh  of  the  minimum  prediction,  and  the  power- 
frequency  capabilities  of  current  transistors  are  orders 
of  magnitude  less  than  forecast.  These  predictions 
therefore  were  not  quantitatively  fulfilled.  They  might 
still  have  served  as  useful  indicators  for  the  planner. 

The  other  two  extrapolations — number  of  transistor 
types,  and  transistor/tube  price  ratios — fell  comfort¬ 
ably  within  predicted  confidence  limits.  Japan’s  1966 
transistor  production  was  72.4  percent  of  that  in  the 
United  States. 

The  application  of  several  different  methods  of  fore¬ 
casting  qualitative  technological  changes  led  to  this 
forecast : 

The  wealth  of  possible  transistor  materials  and  configurations 
that  knowledgeable  experts  could  have  enumerated  in  1957  as 
worthy  of  et^Ioratiun  would  have  given  a  forecaster  grounds  for 
expecting  that  transistor  technology  would  continue  to  advance 
over  the  ensuing  decade  at  a  rate  comparable  to  that  prevailing 
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Figure  VII-5. — U.S.  Transistor  prod 


Figure  VII-6. — TratuUtor  performance  as  a  function  of  JETEC  number. 


through  the  preceding  one.  Shockley’s  expectation,  first  pub¬ 
lished  in  1950,  that  semiconductor  electronics  would  compare  in 
magnitude  and  complexity  with  all  of  the  rest  of  electronics, 
seemed  well  on  its  way  to  fulfillment  in  1957.  The  pattern  of 
applicability,  already  highly  complex  in  1957 . . . ,  could  be  ex¬ 
pected  to  continue  tb  increase  through  additional  applications. 
However,  in  none  of  the  applications . . .  had  the  transistor  be¬ 
come  dominant.  At/ the  point  for  each  of  several  important  ap¬ 
plications — hearing  aids,  computers,  and  domestic  radio  receiv¬ 
ers,  for  example — when  transistor  prices  and  capabilities  made 
them  better  than  competitive  with  vacuum  tubes  and  when  de¬ 
signers  had  acquired  the  skill  necessary  for  designing  with 
them,  design  effort  would  switch  from  tubes  to  transistors. 

These  forecasts  were  borne  out  by  developments 
through  1966; 

Every  one  of  40  hearing  aids  evaluated  in  Consumer  Reports, 
January,  1966,  was  transistorized.  In  the  1957  Spring  and  Sum¬ 
mer  [Sears]  Catalog . . . ,  every  console  phonograph  and  radio- 
phonograph  and  every  portable  radio,  radio-phonograph,  and 
phonograph  listed  is  transistorized.  All  but  4  of  19  table  radios 
are  transistorized _ Only  in  the  television  set  has  the  tran¬ 

sistor  not  taken  over  from  the  tube  in  consumer  electronics. 

The  partial  success  of  these  forecasts  of  transistor 
technology  lends  some  credence  to  the  assessment  of 
the  impact  of  the  transistor  on  the  electronics  industry: 

...  the  U.S.  electronics  industry  continued  to  grow  at  its  his¬ 
torical  rate  of  11  oercent  per  year,  instead  of  “slowing  down,” 
as  it  might  have  had  it  not  been  for  the  transistor.  Consumer 
electronics  products  continued  to  grow  in  sales  volume.  Produc¬ 
tion  in  the  United  State  of  transistorized  radio  receivers  in¬ 
creased  in  volume  by  9  percent  per  year,  but  declined  somewhat 
in  total  value.  Tliis  relatively  slow  rate  of  growth  of  volume  and 
decline  in  total  value  of  receiver  production  is  undoubtedly  due 
in  large  part  to  Japanese  competition. . . . 

The  expectation  that  by  1965  there  would  be  a  half  dozen  or 
less  major  suppliers  of  transistors  in  the  United  States  has  only 
partially  been  borne  out.  Some  41  companies  produce  some 
transistors.  But  not  more  than  a  dozen  of  these  are  major  pro¬ 


ducers,  and  not  more  than  six  account  for  a  majority  of  the 
transistors  produced  in  the  United  States. 

Although  quantitative  data  are  not  available,  there  is  quali¬ 
tative  confirmation  of  the  “forecasts”. . .  of  the  revival  of  stand¬ 
ard-band.  amplitud"  modulation  broadcasting  and  of  the  shift  in 
consumer  repair  sendees  almost  entirely  from  radio  to  tele¬ 
vision.  . . . 

The  major  impact  of  the  transistor  on  industry  in 
general  was  through  its  influence  on  the  cost  and  there¬ 
fore  the  extent  and  distribution  of  automation.  It  was 
easy  to  see  (and  was  foreseen)  in  1957  that  one  major 
impact  of  automation  would  be  on  the  size  of  the 
civil-service  force  and  in  the  distribution  of  demands 
for  skills  in  the  labor  force.  Interestingly  enough,  there 
were  forecasts  about  a  decade  ago  that  automation 
would  produce  large-scale  unemployment  as  its  first 
effect;  but  these  forecasts  were  not  made  by  people  in 
the  electronics  industry.  In  electronics  journals,  the 
forecast  was  that  automation  would  sustain  employ¬ 
ment,  at  least  at  first,  through  its  effect  on  productivity ; 
but  that  it  would  cause  increased  difficulty  with  struc¬ 
tural  unemployment.  These  forecasts,  which  I  embodied 
in  mine,  have  been  fulfilled. 

This  was  about  as  far  as  I  could  go  with  anticipating 
the  social  impact  of  the  transistor.  The  reason  had  to 
do  with  the  fact  that  this  was  a  feigned  forecast,  not  a 
real  one.  I  could  speculate  about  a  wide  range  of  im¬ 
pacts,  from  the  effect  of  the  personal  portable  radio  on 
teenage  culture  to  the  effect  of  Japan’s  economic 
growth,  partly  due  to  the  transistor,  on  Far  East 
stability;  but  I  could  not  demonstrate  that  I  was  not 
just  reading  history  back  into  my  forecast.  So  I  could 
not  go  as  far  in  the  feigned  forecast  as  would  be 
possible  in  an  actual  one. 


CONCLUSION 


In  conclusion,  the  feasibility  of  anticipating  eco¬ 
nomic  and  social  consequences  of  a  major  technological 
innovation  has  in  my  opinion  been  demonstrated.  The 
anticipated  consequences  are  not  exhaustive  or  abso¬ 
lutely  accurate,  but  sufficiently  full  and  precise  to  be  of 
sigiiificant  value  to  the  planner. 

The  process  of  demonstrating  this  feasibility  has 
yielded  the  following  methodological  results; 

1.  Forecasting  is  an  application  of  the  results  of 
empirical  science  on  a  par  with  engineering  and 
medicine. 

2.  There  are  10  generic  types  of  forecasting  methods, 
five  qualitative  and  five  quantitative,  and  they  embrace 


all  methods  described  in  the  literature  that  are  strictly 
forecasting  methods — as  over  against  planning 
methods. 

3.  These  methods  are  not  peculiar  to  technological 
forecasting,  but  are  applicable  to  economic  and  social 
forecasting  as  well. 

4.  They  are  also  applicable  to  answering  the  ques¬ 
tion,  What  will  happen — or  would  have  happened — had 
a  given  innovation  not  come  along? 

5.  Answering  these  questions  is  necessary  in  order 
to  avoid  the  post  hoc,  ergo  propter  hoc  fallacy  in 
as.>iessing  the  stKioeconomic  impact  of  an  innovation. 
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Major  MARTirro.  I  found  your  comments  on  the 
technological  fields  quite  interesting.  I  presume  you 
include  in  this  demographic  forecasting,  which  is  a 
little  different  from  the  others? 

Mr.  Hacke.  Yes,  by  implication  I  answered  that 
query.  The  demographic  forecasting  system  is  one 
model. 

Dr.  Johnston.  1  think  you  probably  covered  this 
technique  as  well,  but  I  wish  I  had  been  able  to  an* 
ticipate  some  of  the  things  about  your  paper.  You 
mentioned  the  qualitative  techniques — I  guess  it  was 
this  trend  correlation  technique.  Have  you  tried  the 
lead-lag  indi'  ators  for  shortrun  purposes?  That  is,  i*^ 
you  have  a  technological  factor  which  is  a  close  relative 
of  one  and  the  other  is  going  ahead.  Have  you  had  any 
luck  with  this? 

Mr.  Hackc.  I  haven’t  used  it;  just  analyzed  it.  I 
would  include  this  lead-lag  as  a  form  of  trend  cor¬ 
relation. 

Mr.  Irwin.  I  have  a  question  of  detail  regarding 
trends  correlation.  This  is  with  respect  to  using  re- 
gresu'on  analysis,  either  simple  or  -nultiple  regression 
and  concerns  the  case  of  extreme  values.  The  ones  I 
refer  to  ere  related  to  employment  and  population 
change  by  State.  In  all  the  data  I  have,  1940-1%0,  arc 
three  Sta'es  that  drove  very  fast;  California,  Arizona, 
and  Florida,  and  a  couple  of  States  arc  losing  like 
crazy:  West  Virginia  and  Pennsylvania.  Usually  the 
other  45  States  or  so,  are  in  the  center  and  very  often 
are  not  correlated.  It  looks  like  Denis  Johnston’s  chart. 
But  with  those  five  States  that  spread  in  a  line  like  the 
moons  floating  around  Jupiter  you  get  very  good  cor¬ 
relation  value  when  you  put  all  the  numbers  in,  like 
0.98,  0.97..  Take  them  out  and  it  goes  down  to  about 
0.6.  Is  there  a  recommended  technique  for  dealing  with 
this?  What  would  you  suggest? 

Mr.  Kacke.  What  are  you  correlating  with? 

Mr.  Irwin.  I  usually  have  migrat'on  as  my  dependent 
variable.  For  example,  take  regional  sh’ft  in  the  share 
of  each  industry  by  State  in  the  forties  snd  correlate  it 
with  net  migration  in  the  forties  and  you  get  very  fine 
correlation  if  you  leave  all  the  States  in,  but  Califorria 
is  always  there,  Arizona  and  Florida  way  o.  *,  every¬ 
thing  going  up,  and  on  the  other  side,  everything  going 
down. 

Mr.  Hacke.  I  think  what  you  are  saying  is  these  five 
States  are  obvious  in  their  advance,  but  if  you  include 
all  the  other  Slates  in  you  a  higher  correlation  than 
if  you  Include  them  out.  Is  that  right  ? 

Mr.  Irwin.  I  didn't  think  of  doing  that.  I  said, 
“Look,  I’ve  got  to  sec  this  vslue  about  these  Stales,  so 
I  pulled  them  out.” 


Mr.  Hacke.  You  pulled  these  five  States  out  and  ran 
the  correlation  again?  What  happened? 

Mr.  Irwin.  It  dropped  way  down.  The  point  ia; 

Si  posing  I  go  to  ’he  next  step  and  use  my  multiple 
regression  eqw.-’ion  or  a  simple  regression  equation  as 
a  predicter.  I  can  say  for  sure  I  am  going  to  predict 
these  five  dynamic  States  pretty  well,  assuming  the 
future  will  be  like  the  past,  but  what  about  those  in 
the  middle?  Granted  they  are  not  changing  as  much 
and  I  can’t  make  as  bad  a  mistake  because  you  are  not 
actually  forecasting  change,  bu-  it  puts  the  use  of 
regression  equations  in  question  and  this  is  the 
question  I  am  raising. 

Mr.  Hacke.  No,  I  don’t  think  it  does.  You  are  saying 
essentially  is  you  don’t  have  enough  data  on  the  migra 
tion  of  these  in-the-middle  States  to  make  a  medium 
for  prediction.  We’re  going  to  have  inmigration  or 
outmigration.  That’s  about  all  I  can  say,  however  you 
have  to  do  a  more  detailed  analysis.  It  is  not  the  form 
of  the  regression  that’s  at  fault.  It  is  just  that  you  need 
more  data. 

Mr.  Irwin.  We  took  a  survey  for  the  Ofiice  of  Eco¬ 
nomic  Opportunity.  Collected  data  and  found  one 
datum  missing.  We  have  an  imputation  procedure 
whereby  we  impuis  characteristics  to  a  person  on  the 
basis  of  six  or  eight  other  characteristics.  However, 
the  imputation  procedure  did  a  bad  thing  on  pro¬ 
fessional  policemen.  They  used  what  they  called  a  hot 
deck”.  People  arc  coming  and  going  .ii  the  imputation, 
then  when  characteristics  are  missing,  you  look  in  the 
hot  deck  to  sec  who  was  the  last  one  in  that  followed 
these  characteristics  and  the  implication  is  a  geographic 
connection,  which  happened  to  be  haywire,  so  this 
full-time  professional  policeman  of  fairly  high  talent 
received  a  salary  of  $100  a  year  the  consultants 
working  with  the  data  were  extremely  unhappy  and 
said  it  was  going  to  ruin  thr:;-  regression  equation. 
Sh^uId  we  use  a  tool  where  a  crazy  value  will  ruin  it 
especially  when  you  start  getting  complex  models  that 
use  stepwise  multiple  regression  with  snv  bO  variables, 
and  makes  a  choice  on  an  1  -lest  of  every  variable  when 
you  don't  know  what  goes  into  it? 

Mr.  Hacke.  Nothing  comes  out  of  a  thing  lietter  than 
what  goes  in. 

Mr.  Irwin.  There  is  nothing  I  can  do  about  the  it;put. 

Mr.  Hack.e.  a  committee  is  a  group  of  people  who 
individually  do  nothing  but  when  thev  meet,  they 
decide  .lothing  can  he  done.  me  say  this;  I  think 
one  of  tha  major  advantage*  of  tech  forecasting  is  how 
much  you  know  about  it. 

Mr.  Smitii,  In  answer  to  Mr,  Johnson,  we  have  done 
some  forecasting  using  trend  correlation  and  coin- 
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poncnts,  I  guess  you’d  call  it,  component  trend  corre¬ 
lation  and  I  have  a  report  that  was  published  before. 
It  is  an  old  report  but  shows  that  using  this  component 
trend  correlation  that  in  the  past  year  and  a  half  since 
we  accomplished  work,  our  forecasts  seem  to  be  quite 
valid  with  this  particular  method.  I've  given  the  report 
to  Mr.  Hacke  and  I  am  sure  he  would  be  glad  to  have 
you  look  at  some  of  these  curves. 

Major  SwETT.  I  have  a  definitional  question.  You 
were  describing  extrapolation  as  inductive.  Would  it 
be  correct  to  say  the  fitting  of  a  curve  by  regression 
technique  is  the  inductive  process  whereby  you  would 
have  a  hypothetical  function,  but  then  the  extrapola¬ 
tion  itself  is  a  deduction  based  on  the  hypothesis? 

Mr.  Hacke.  I’ve  been  around  this  semantic  merry- 
go-round  a  couple  of  times  and  don’t  know  a  better 
way  to  say  what  I  want.  Of  course,  drawing  the  line  or 
making  a  particular  forecast  is  a  deduction  from  the 
mathematical  expression  you  generate,  but  generating 
the  extrapolation  itself,  the  mathematical  curve  itself 
is  an  inductive  process.  Looking  at  the  data,  you  say, 
“Gee,  it  looks  like  this.  I  assume  the  generality  is  eoing 
to  be  as  follows.” 

Major  SwETT.  What  occurs  is  a  hypo' ho-,  is  bases  on 
the  data  points. 

Mr.  Hacke.  Yes. 

Dr.  Linstone.  The  sociological  consequences  of  ma¬ 
jor  technological  innovation  brings  to  mind  something 
we  experienced  during  the  Mirage  studies.  The  first  one 
in  particular  where,  you  might  say,  military  technologv 
invaded  the  military  area  and  its  impact.  An  analysis 
of  the  rapid  deployment  with  a  Lanchester-type 
analysis  has  shown  the  implicatior.  was  quite  evident 
that  the  faster  you  go  into  a  situation  and  nip  it  in  the 
hud,  the  less  force  is  needed.  As  I  say,  this  was  kind 
of  in  our  innocence.  Tm  more  and  more  skeptical  that 
such  a  result  is  meaningful,  because  this  is  a  socio¬ 
implication  or  psychologica!  implication.  It  doesn’t 
always  work  the  way  theory  predicts.  All  that  happen.^ 
is  you  fend  to  go  into  more  places.  Nothing  apparently 
rerluces  the  deployment  of  troops. 

Mr.  Hacke.  You  go  into  more  places  with  less  and 
less  until  you  end  up  with  nothing. 

Mr.  I.I.VSTO.NE.  You  have  the  desire  and  the  capability 
to  use  iiKirr  and  never  therefore  economiie  with  less. 
I  don’t  know  if  it’s  true  hut  it  is  kind  of  a  feeling  now- 
adavs.  Tliis  l>eautiful  argument  for  more  airplanes,  etr. 

Major  Martino.  1  think  Mr,  1, instone’s  quesjion  is 
the  .same  reganling  social  implication.  He  is  suggesting 
the  naive  prediction,  from  a  better  militarv  capabih  v. 
is  that  von  do  the  j-ih  with  less  pe<sple  ami  a  move 
.sophistiraled  implication  is  that  you  will  contract  to 


do  more  jobs  and  still  use  the  same  number  of  people. 
Would  you  care  to  comment  on  either  df  these  pr-  lie 
tions? 

Mr.  Hacke.  I  think  there  has  to  be  recognition  and 
all  of  us  in  forecasting  are  rhuch  *co  fond  of  linearity. 
If  one  man  can  build  a  house  in  30  days  can  30  men 
build  a  house  in  one  day?  There  is  an  optimum  beyond 
which  this  total  force  would  increase.  I  don’t  know 
wheilier  this  is  what  you  are  suggesting,  but  there  is 
an  area  of  compensatory  magnitude  in  many  situations 
that  operates  to  reduce  tl.s  impact  of  the  socioeco¬ 
nomics  of  many  technological  innovations.  If  we  put 
the  effort  on  miniaturization  and  riggediiation  in 
vacuum  tubes  that  we  have  p  ‘  >n  transistors,  I  don’t 
think  our  electronics  technology  would  be  greatly 
different  from  that  which  we  have  today. 

Dr.  Linstone.  There  is  something  that  affects  fore¬ 
casting.  We  rarely  see  any  document  or  discussion  of 
negative  needs.  What  do  I  mean  by  negative  needs? 
We  all  talk  about  what  new  thing  will  be  added. 
I  don’t  knew  whul  pyschological  factors  are  operating 
but  I  think  there  is  a  real  problem.  People  are  not  going 
to  give  up  anything.  I  think  ihut’a  more  true  than 
people  recognize  as  the  negative  forecasts. 

Mr.  Hacke.  One  example  which  illustrates  what  1 
think  you  mean  is — there  was  an  ingenious  new  type 
of  vacuum  tube  developed  about  8  years  ago  with  a 
4-year  lifespan  and  that  was  a  low-anode-voltage 
vacuum  tube.  It  worked  fine  as  an  RF  amplifier  in  con¬ 
junction  with  transistors  in  the  audio  section.  It  was 
used  for  a  while  and  then  suddenly  trans'stoi 
technology  passed  it  by. 

Dr.  Linstone.  It  isn’t  precisely  like  that.  I’m  not 
talking  about  a  new  item  that  might  not  be  the  best. 
For  example,  in  the  Delphi  there  was  forecasting  of 
breakthroughs.  These  are  new  advances  the  positive 
.side  of  the  ledger,  so  to  speak.  1  would  like  to  sec  a 
corresponding  list  ol  'hings  to  discard  or  things  we 
won’!  have. 

Mr.  Hacke.  Are  you  talking  about  things  that  have  a 
di.sfunrtion.  a  'iisutility? 

Dr.  Ijnstone.  Yes. 

Major  SwETT,  This  discu.ssion  belongs  in  a  class  of 
problems  involving  waste,  a.cd  accumulation  genereled 
hv  the  existing  svstem. 

Dr.  Linstone.  When  vou  have  supply  over  demand.  - 
vou  have  more  of  something  around,  then  vou  have  a 
I’isutililr  function  and  it  is  the  same  problem  with 
rliminating  atomic  wastes,  vmo^ ,  and  bacteria;  and 
o.ir  own  >*,;«te  that  vou  can’t  get  rid  of,  this  is  an  area 
that  has  l>ecr)  oractically  unaddrrssrd  s  '  far.  This  prob- 
!•-  ••,  came  up  at  one  of  the  Howie  btiards,  O-e  of  the 
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obvious  questions  Mr.  McNamara  wanted  answered 
was:  What  don’t  we  need  and  what  can  we  get  rid 
of? — this  caused  more  agony  than  the  whole  new 
concept.  It  was  decided  a  narrow  gage  locomotive  was 
something  you  don’t  need. 

Major  Martino.  This  is  something  about  vested 
interest  ai.d  all  that  implies. 

Dr.  Chacko.  I  would  like  you  to  comment  on  some¬ 
thing  which  is  related  to  both  questions  and  problems 
that  have  been  raised  recently.  As  you  know.  Congress¬ 
man  Daddario  introduc:ed  a  bill  for  technology  assess¬ 
ment.  He  told  me  Congress  bought  off  on  the  idea  of 
supersonic  transport  without  much  critical  review. 
Then  SkI  showed  there  will  be  a  sonic  boom  when  the 
supersonic  transport  travels  over  landmasses  as  opposed 
to  water  surface.  This  was  not  practical,  the  question 
arises;  From  your  study  of  this  transistor  radio  or 
allied  work,  is  there  any  methodology  we  could  see  that 
will  address  a  more  general  question  of  technology 
assessment,  or  fallouts,  of  any  innovation?  Do  you 
have  any  words  of  wisdom  to  offer? 

Mr.  F.vcke.  In  the  first  place,  we  were  trying  to 
demonstrate  a,  method  or  develop  a  demonstrated 
method,  rather  than  produce  a  study  that  may  have 
some  words  of  wisdom  to  come  from  this  kind  of 
question.  But  from  other  work  that  has  been  conducted 
at  SRI  on  educational  policies,  research  centers,  I  am 
speaking  for  myseH  as  a  participant  in  the  study  and 
not  for  the  study,  this  series  of  questions  relates  to  the 
question  of  values.  Wlien  you  get  into  the  area  of 
human  or  humane  values  it  is  easy  fcr  things  to  become 
superficial.  There  has  been  so  much  superficial  writing 
and  speaking  on  this  topic.  A  second  point  regarding 
Daddario’s  proposal  was  for  a  technology  evaluation 
adjunct  to  Congress.  We  were  saying  that  undoubtedly 
the  time  has  long  since  passed  when  we  needed  a  staff 
in  Congress  to  give  them  a  way  of  acquiring, 
at  least  as  a  planner  or  decisionmaker,  some 
insight  into  the  technological  process  and  a  means  for 
evaluating  the  approach  goals  and  objectives  through 
the  use  of  technology.  The  new  scientists  have  an  idea 
of  .setting  up  what  amounts  to  a  technocracy  to  formu¬ 
late  goals  and  plans  but  it  is  ideologically  dangerous. 
At  least  in  a  democracy  the  feedback  process  from 
goals  to  plans  should  include  the  populace.  I  think 
Congress  has  the  responsibility  and  it  cannot  escape. 
They  can  have  a  staff  but  its  line  function  is  goal 
formulation  and  plan  adoption. 

Or.  Dalkey.  You  partly  answered  my  question  bul  I 
would  Uke  to  get  it  on  the  record.  There  is  a  rather 
amusing  pataoox  particularly  in  the  use  of  growth 
curves  to  forecast  technological  developments  and  at 


the  same  time  assess  the  impact  of  those  technologicd 
developments.  You  said,  you  thought  you  could  make 
an  as-if  kind  of  assessment  of  what  would  have  hap¬ 
pened  if  the  transistor  had  not  been  put  in  the  tech¬ 
nological  stream.  If  we  take  this  notion  that  lies  behind 
the  use  of  the  growth  curve,  which  is.  that  a  form  of 
technology  will  arise  and  keep  that  cutve  going,  if  the 
transistor  hadn’t  been  invented,  then  presumably 
something  else  would  have  kept  the  growth  curve 
going;  in  which  case  you  might  s?,y  the  impact  of  the 
transistor  is  to  the  first  order  of  approximation  null, 
because  if  it  hadn’t  been  invented  something  else  would 
have  taken  its  place. 

Mr.  Hacke.  I  would  not  make  exactly  that  assump¬ 
tion  but  go  ahead. 

Dr.  Dalkey.  There  is  a  slightly  amusing  paradox 
because  if  you’  go  back  in  the  past  end  ask  the  question : 
What  would  have  happened  if  such  and  such  hadn’t 
been  invented?  You  would  be  in  an  interesting  position 
of  having  to  invent  what  would  have  been. 

Mr.  Hacke.  You  are  precisely  in  the  same  predica¬ 
ment  as  the  forecaster  who  cannot  imagine  the  inven¬ 
tions  or  innovations  that  are  actually  going  to  be  made. 
The  railroad  system  and  the  way  it  has  been  developed, 
for  example,  might  well  have  been  an  invention  that 
never  would  have  been  needed  because  of  the  horseless 
carriage.  I  don’t  know,  I’m  just  a  forecaster. 

Dr.  Dalkey.  If  you  are  trying  to  evaluate  the  impact 
of  a  named  item  you  can  say  the  impact  of  any  given 
named  item  is  essentially  zero  because  if  that  named 
item  hadn’t  arisen,  something  would  have  taken  its 
place. 

Mr.  Hacke.  Let  me  use  the  transistor  as  an  example 
and  incidentally,  the  choice  of  the  transistor  came  as 
a  result  of  a  study  of  a  fairly  large  number  of  nomi¬ 
nated  innovations  because  it  was  completely  defined. 
It  was  fairly  unambiguous.  A  border  has  to  be  drawn 
between  other  solid  state  devices  and  the  transistor, 
but  otherwise  you  have  a  straightforward  job.  There 
are  many  innovations  that  are  hard  to  define.  Specifi¬ 
cally  with  the  transistor  which  is  noted  in  the  report, 
I  think  you  can  say:  had  the  transistor  not  been 
developed,  true  technology  would  have  continued  to 
develop  at  approximately  the  rate  it  did  before  the 
transistor  came  along,  therefore,  some  compensatory 
mechanism  would  have  operated  with  penalties  in  terms 
of  weight,  cost,  anJ  power  consumption.  The  forecast 
of  the  transistor  is  not  nil.  You  say  something  else 
would  have  been  invented.  I  can’t  answer  that;  I  can 
only  carry  it  as  far  as  1  can  investigate  the  alternative. 

Dr.  Dalkey.  I  don’t  want  to  push  it  too  far  but  if 
you  are  going  to  apply  things  like  growth  curves  in  the 
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future,  wouldn’t  you  ’do  willing  to  apply  them  to  an  curves,  everybody  gets  unhappy.  Will  you  assume  a 

uncertain  past?  mechanism  of  change  and  investigate  mathematically 

Mr.  Hacke.  Yes.  what  that  method  means?  These  have  been  derived  by 

Dr.  Dalkey.  It  seems  this  would  be  rather  an  amus-  analogy,  from  bacteria,  and  I’m' not  happy  about 

ing  exercise.  bacteria. 

Mr.  Hacke.  I  think  methodologically  also  you  arc  Colonel  Davis.  I  took  the  opportunity  after  your 
on  the  same  ground  trying  to  forecast  what  will  happen.  lecture.  Dr.  Dalkcv,  to  show  where  it  seemed  feasible 

Colonel  Vaughn.  You  disci  ssed  10  different  methods  to  fit  a  growth  cu  'i  t  v>i  logistic  curve  to  the  outcome  of 

but  I  imagine  none  of  them  are  used  in  their  pure  and  a  prediction  by  a  g.'  Jup  of  Delpl  1  experts,  and  since 

sole  state;  that  there’s  always  some  mixture.  all  of  the  Delphi  curviis  have  this  characteristic,  one 

Mr.  Hacke.  There  are  a  couple  that  are  used  alone,  might  think  it  possible  to  do  that  with  others.  Conse- 

i  or  in  a  fairly  pure  state.  One  is  the  use  of  experts,  quently,  the  results  of  Delphi  people  may  lead  to  a 

I  inductive  trend  determination,  induction  and  extrapola-  phenomena  which  can  be  described  or  a  description  of 

I  tlon.  These  occur  to  everybody  to  try  first.  I  think  I  can  »  phenomena  that  could  be  characterized  by  growth 

i  anticipate  what  you  are  going  to  suggest  but  you  should  curves.  1  wanted  to  remind  you  so  you  won’t  be  too 

I  use  as  many  different  methods  as  possible  as  a  check.  unhappy  with  gro^yth  curves  in  the  event  they  would 

I  Colonel  Vaughn.  I  was  wondering  if  that  is  good  characterize  mathematically  some  of 

policy?  It  appears  to  be  good  technique  for  a  person  Delphi  results. 

to  use  one  against  the  other  and  hopefully  use  several  Dr.  Dalkey.  I  hope  I  didn’t  give  the  impression  T 
to  check  against.  was  unhappy  with  growth  curves.  On  the  contrary, 

Mr.  Hacke.  Yes,  you  can.  growth  curves  seem  to  have  a  great  deal  of  validity 

Colonel  Vaughn.  We  sponsored  a  study  at  Syracuse  have  a  life  of  their  own.  This  is  what  results  in 

i  University,  which  was  basically  an  expert  report  bound  Ae  paradox. 

\  together  and  more  or  less  correlated.  At  least  they  all  T)r.  Chacko.  You  asked  for  another  word  for 

f  used  the  same  language;  but  I  can  imagine  that  each  “growth.”  I  have  used  “life  cycle  curves.” 

.  person  in  his  own  disciplinary  area  probably  used  all  Major  Martino.  That  is  a  good  terminological 

j  10  and  found  more.  suggestion. 

Mr.  Hacke.  My  experience  from  use  of  experts  and  Dr.  Chacko.  One  of  the  major  technological  innova- 
;  I  have  been  an  expert  in  a  forecast  and  have  seen  these  tions  is  probably  a  better  nuclear  detection  device  and 

I  things — many  of  the  books  you  read,  from  H.  G.  Wells  when  we  sign  a  test  ban  treaty,  we  will  know  who  is 

I  on,  are  expert  opinion  books — is  that  the  expert  gen-  cheating.  The  technical  means  to  detect  this  should  be 

!  erally  operates  by  using  historical  analogy  and  in-  available.  Should  this  come  about  and  lead  to  a  de¬ 
ductive  trend  determination.  The  expert  uses  the  escalation  of  the  current  armaments  race,  one  of  the 

method  and  his  methods  are  subject  to  analysis  as  are  major  problems  is  that  of  economic  impact.  I  am  sure 

j  anybody’s.  you  have  studied  the  report  that  was  published  about  2 

Mr.  Smith.  Colonel  Vaughn,  on  using  several  dif-  d  years  ago  showing  the  probable  impact  on  several 
;  ferent  methods,  from  the  experience  we  had- -we  used  firms  largely  dependent  on  defense  contracts.  I  suggest 

’  growth  curves,  trend  correlation  and  experts  and  put  tb's  will  be  a  major  technological  innovation  and  will 

together  a  tech  forecast  about  a  year  ago.  We  found  have  serious  major  socioeconomic  consequences, 

almost  complete  disagreement  between  the  experts.  Would  you  comment  on  how  you  look  at  this  problem? 

growth  curves  and  trend  correlation,  almost  100  per-  Mr.  Hackf.  Although  this  is  not  a  quarrel  with  the 
cent  in  every  forecast  We  finally  resolved  it  by  taking  problem  you  pose.  1  don’t  think  the  question  is  a  tech- 

the  growth  curves  and  trend  correlation,  because  we  nological  innovation.  However,  an  economic  and 

could  document  it  against  many  opinions,  and  found  political  innovation  is  there  We  have  adequate  test 

experts  quoting  historical  analogy  to  contradict  our  devices  for  all  prartical  purposes.  One  of  the  things 

curve*.  You  can’t  mix  the  three,  not  in  our  ex|)erience  1  think  you  are  asking  is;  that  this  would  be,  as  all 

fore-asts  have  to  Ive,  a  conditional  fi.rev'usl  based  on  the 
Colonel  VaI'CIin.  It  might  be  interesting  to  try  the  decisions  made  b,-  significant  actvirs  in  the  situation. 
Delphi  technique.  The  United  States  will  undoui)te<i!y  Adopt  '*otac  pro- 

Mr.  IfAf.KK.  I  am  unhappy  af)out  l\\c  tillr  growth  gr.nms  of  opp  kind  or  anothrr  to  amrliofAtc  ih eirploy- 

curves.  If  you  discuss  growth  curves  vou  also  have  to  ment  impact  and  the  like  of  partial  or  total  deescaU- 

talk  about  decay  curves,  but  if  you  discuss  decay  tion.  It  will  depend  -the  forecaMer  should  give  the 
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planner  in  the  light  of  the  innovation  what  the  probable 
coiisequenceii  of  the  visible  alternatives  would  be  so 
that  he  could  make  a  selection,  and  could  exercise 
judgment  to  decide  which  and  what  strategy  to  use. 

Major  SWETT.  I’m  not  sure  correlation  would  be  of 
interest.  The  Delphi  pattern  of  convergence  through 
reiteration  shows  a  split  logistics  curve  normally. 

Mr.  Hacke.  What  do  you  mean  by  split  curve? 

Major  SwETT.  A  split  curve  on  both  sides.  If  you  are 
looking  at  the  convergent  cone,  each  half  of  it  looks 
like  a  logistics  curve  normally.  The  technoingical  de¬ 
velopment  curve  oftentimes  dso  shows  a  logistics 
shape.  Psychologists  use  the  same  term  and  found  it 
repeatedly  in  their  learning  phenomena.  They  called  it 
a  learning  curve.  It  seems  both  pyschological  and  tech¬ 
nological  progress  show  a  characteristic  of  learning. 

Mr.  Hacke.  I  wrote  a  section  of  this  report  which 
attempted  to  demonstrate  this.  The  normal  curve  of 
error  can  be  deduced  from  very  general  mathematical 
assumption.  Therefore,  it  has  a  wide  applicability  in 
its  description  and  I  think  the  same  general  assump¬ 
tions  can  be  made  by  the  nature  of  technological 
progress,  and  will  yield  the  same  kind  of  curve.  It  is 
not  accidental  that  the  estimates  people  make  of  an 
imperfectly  known  quantity  in  the  learning  curve  and 
many  growth  curves  are  all  the  same  shape. 

Dr.  Johnston.  I  remember  in  the  courses  where  we 
have  anything  normally  distributed  we  accumulate  the 
normal  distribution  by  logistics  and  this  was  discovered 
in  the  1920’s  again  and  found  to  apply  to  a  vast  range 
of  phenomena  and  it  is  dangerous  to  forecast  from  it 


because  it  is  difficult,  on  a  time  scale,  locating  yourself 
on  that  curve. 

Mr.  Hacke.  Yes,  until  you  get  to  the  inflection  point, 
you  don’t  have  enough  information  as  to  where  you  are 
going,  and  then  it  is  too  late. 

Major  Martino.  The  same  is  true  of  Zipfs  Law. 
Beeause  of  some  fundamental  mathematical  things  it 
turns  out  to  have  wide  applicability.  But  you  run  into 
the  same  kind  of  a  trap.  You  never  know  whether  this 
is  one  of  those  cases. 

Mr.  Hacke.  Until  it  is  too  late. 

Major  Martino.  You  mentioned  Terman’s  forecast 
made  in  1932  regarding  fr^’quency  modulation.  The 
reason  Terman  was  wrong  in  this  case  was  in  the  1920’s 
people  first  started  looking  at  frequency  modulation 
and  not  understanding  the  situation.  They  believed  one 
could  transmit  the  same  amount  of  information  over 
a  narrower  bandwidth  which  would  be  wonderful. 
Finally,  they  understood  this  was  not  so.  that,  in  fact, 
FM  would  take  more  bandwidth  or  as  much.  All  the 
experts  soured  on  FM.  They  were  completely  against 
it  because  it  would  mA  give  the  advantage  originally 
hoped  for.  The  advantage  of  FM  lies  in  another  field.  It 
allows  you  to  "“t  rid  of  noise  and  the  problem  is  that 
Terman  was  answering  the,  wrong  question. 

Mr.  Hacke.  Yes. 

Major  Mart'NO.  You  have  to  be  aware  of  this  prob¬ 
lem  in  dealing  with  an  expert.  Make  sure  he  is  answer¬ 
ing  the  right  question. 

Mr.  Hacke.  I  agree. 


One  of  the  most  widely  used  forecasts  currently  being  prepared  by  the  U.S.  Govern¬ 
ment  is  that  of  the  weather  While  weather  forecasting  has  been  practiced  by  farmers, 
sailors,  and  other  for  centuries,  the  use  of  scientific  tools  for  forecasting  is  a  com- 
parati"eiy  recent  innovation.  The  problem  of  the  weather  forecaster,  since  the  intro¬ 
duction  of  scientific  methods,  has  been  that  he  can  have  acce;  '  to  more  data  than  he 
can  possibly  use,  in  the  time  he  is  allowed  to  make  his  force  The  advent  of  the 
digital  computer  is  already  changing  this  situation,  however.  Commander  Hamilton’s 
piper,  on  numerical  weather  forecasting,  presents  some  of  the  problems  of  a  forecaster 
with  «  plethora  of  data,  but  whose  theory  about  the  meaning  of  the  data  is  not  always 
adequate  to  the  (ask  of  preparing  reliable  forecasts. 


— Editor 


NUMERICAL  WEATHER  FORECASTING  TECHNIQUES* 


Comdr.  G.  D.  Hamilton,  U.S.  Navy,  Float  Numerical  Weather  Facility,  Monterey,  Calif. 


The  development  of  numerical  methods  of  weather 
analysis  and  forecasting  is  one  of  the  most  significant 
and  spectacular  advances  in  the  field  of  meteorology 
in  its  long  history. 

The  numerical  prediction  process  attempts  to  fore¬ 
cast  atmospheric  motions  by  formulating  them  in 
mathematical  terms.  To  accomplish  this,  Sir  Isaac 
Newton’s  laws  of  motion,  which  were  developed  for 
bodies  or  particles  subject  to  impressed  forces,  had 
first  to  be  applied  to  a  fluid  medium.  This  was  done  in 
1755  by  the  German  mathematician,  L.  Euler;  however, 
it  was  not  until  the  latter  half  of  the  19th  century  that 
Euler’s  hydrodynamical  equations  were  applied  to  the 
study  of  the  atmosphere.  A  treatise,  published  in  1888 
by  H.  von  Helmholtz,  a  scientific  genius  of  his  period, 
applied  the  hydrodynamic  equations  to  the  atmosphere. 

It  might  be  wondered,  therefore,  why  our  interest  in 
solving  these  equations  is  so  belated. 

The  answer  is  probably  twofold.  In  the  first  place, 
the  equations  are  simply  so  difficult  to  solve.  In  mathe¬ 
matical  terms,  the  difficulty  is  one  of  solving  a  general 
boundary — and  initial — value  problem  for  a  simul¬ 
taneous  system  of  six  nonlinear  partial-differential 
equations  in  three  dimensions.  Even  today,  there  are 
no  known  methods  by  which  the  solutions  of  such 
equations  can  be  related  explicitly  to  general  boundary 
and  initial  conditions. 

Exact  analytic  methods  failing,  the  most  satisfactory 
course  would  have  been  to  s<>lve  the  equations  by  purely 
numerical  methods.  These,  however,  were  not  fullv 
developed  until  early  in  the  20th  century  and,  .n  any 
case,  would  have  required  an  enormous  volume  of 
computation. 

The  second  and  probably  more  important  obstacle  to 
the  early  development  of  dynamical  weather  prediction 
was  the  sparse  meteorological  data.  Tlie  dynamical 
meteorologist  of  that  day  did  not  really  know  what  kind 
of  phenomenon  he  had  to  explain,  and  could  not  fuliv 
test  any  theory  that  he  might  propose  to  account  for 
the  behavior  of  the  large-scale  weather  disturbances. 

The  realisation  that  the  general  hydrodynamical 


equations  could  be  solved  in  principle,  and  if  necessary, 
by  purely  numerical  ntetliods  and  sheer  brute  force, 
appears  to  have  first  occurred  to  L.  F.  Richardson,  a 
highly  original  British  meteorologist-mathematician- 
economist-statistician  who  also  had  a  lively  interest  in 
the  new  finite-difference  methods.  During  World  War  I, 
between  ambulance  trips  to  the  front,  he  designed  and 
carried  through  a  finite-difference  scheme  for  solving 
the  nonlinear  hydrodynamical  equations  for  meteor¬ 
ological  purposes — the  first  genuine  attempt  at  dynami¬ 
cal  weather  prediction.  Anticlimatically,  Richardson’s 
results  disappeared  in  the  general  confusion  of  the  war, 
together  with  the  computations  he  carried  out  labori¬ 
ously  by  hand  over  many  months.  They  were  later 
found  under  a  coal  heap  and  published  in  1922. 

Richardson’s  experiment  was  not  an  unqualified  suc¬ 
cess,  since  his  calculations  predicted  that  large-scale 
weather  disturbances  would  travel  at  about  the  speed 
of  sound  and  in  the  wrong  direction.  In  addition,  he 
estimated  that  it  would  take  64,000  human  computers 
operating  simultaneously  just  to  predict  weather  as  fast 
as  it  happened.  Numerical  prediction  could  not,  there¬ 
fore,  become  a  practical  reality  until  the  development 
of  high-speed  electronic  computing  machines. 

Another  very  important  factor  in  the  development  of 
numerical  prediction  was  the  establishment  of  an  inter¬ 
national  network,  of  upper  air  observing  stations.  Our 
present  day  network,  while  still  sparse  over  ocean  and 
tropical  areas,  is  fairly  dense  in  many  land  areas. 
Furthi  rinore,  the  observations  are  not  confined  to  the 
earth’s  surface  but  data  throughopt  most  of  the  vertical 
extent  of  the  atmosphere’s  mass  is  obtained  by  balloon 
.soundings.  Thus,  a  three-dimensional  picture  of 
atmospheric  motion  is  obtained. 

By  the  end  of  World  War  II,  the  stage  was  set  for 
rapid  progress  in  numerical  prediction.  The  design  of 
the  ner'cssary  high-speed  computers  was  still  in  prog¬ 
ress  when  a  nunserical  prediction  group  was  formed  in 
l')46  under  the  leadership  of  J.  von  Neumann  at  the 
Institute  for  .Advanced  Study  at  Princeton.  Sponsored 
by  the  Office  of  Naval  Research,  this  group  simplified 
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the  hydrodynamical  equations  under  certain  limiting 
conditions,  so  that  they  could  be  solved  with  the  aid  of 
computers.  Several  solution  schemes  or  models  were 
developed,  tested  and  successfully  applied  to  predicting 
large-scale  atmospheric  motions. 

There  were  two  serious  defects  in  Richardson’s  work. 
First,  it  is  a  matter  of  experience  that  the  atmosphere  is 
always  very  close  to  a  state  of  mechanical  equilibrium; 
i.e.  the  horizontal  pressure-gradient  force  is  almost 
exactly  in  balance  with  the  Coriolis  force,  and  the 
vertical  force  of  buoyancy  is  almost  exactly  balanced 
by  the  virtual  gravitational  force  (the  resultant  of  the 
earth’s  centrifugal  force  and  pure  gravitational  forces) . 

Accelerations  of  air  are  dependent  upon  the  small 
difference  in  these  forces  and  pressure  gradients  and 
winds  cannot  be  measured  with  the  required  accuracy. 
Second,  the  hydrodynamical  equations  are  general  and 
include  high-speed  sound  and  gravity  waves  as  well  ai 
the  meteorologically  important  long  atmospheric  waves. 
This  may  lead  to  computational  instability  and  the 
solution  will  blow  up  unless  the  time  increment  is  less 
than  the  time  required  for  sound  waves  to  travel  the 
gnd  distance  over  which  the  finite  differences  are  taken 
to  approximate  derivatives.  To  provide  sufficient  reso¬ 
lution  of  the  pressure  field,  the  grid  distance  must  be 
taken  in  the  neighborhood  of  200  km.,  which  therefore 
requires  a  time  increment  of  about  10  minutes. 

From  physical  considerations,  there  is  little  reason  to 
expect  that  such  high-frequency  waves  would  have 
much  significance  in  large-scale  meteorological  phe¬ 
nomena,  and  it  is  therefore  desirable  to  filter  out  these 
waves.  When  this  filtering  has  been  accomplished, 
larger  time  increments  (of  the  order  of  an  hour)  are 
permissible  for  the  individual  steps  of  the  forecast. 

This  is  the  type  of  filtered  model  that  was  first  com¬ 
puted  by  von  Neumann’s  group  in  1950  on  the  ENIAC 
computer.  The  modified  hydrodynamical  equations 
describe  a  relatively  simple  model  atmosphere  (bare- 
tropic)  and  it  has  rather  serious  shortcomings.  It 
implies  that  the  number  and  intensity  of  cyclonic  and 
anticyclonic  vorticities  cannot  change  and  accordingly 
cannot  predict  the  formation  and  growth  of  new  dis¬ 
turbances.  Thus,  in  order  to  deal  with  these  problems, 
more  general  models  have  been  synthesized.  Many  of 
these  models  Have  been  developed,  tested  and  shown 
that  relatively  crude  methods  of  dynamical  prediction 
were  as  effective  as  subjective  techniques  in  the  hands 
of  a  skilled  forecaster  and  were  accurate  enough  to 
justify  putting  them  into  practice.  Today,  computing 
machines  have  taken  over  major  segments  of  the 
routine  weather  fore('arting  job. 

With  the  advent  of  larger  and  faster  computers,  more 


sophisticated  models  appeared  which  could  predict  the 
formation  and  strengthening  of  new  pressure  centers. 
However,  they  rarely  predicted  the  occurrence  at  the 
right  times,  at  the  right  places  or  at  the  correct  rate. 
Discouragingly,  the  more  complicated  models  have  not 
shown  significantly  better  skill  than  the  barotropic 
model. 

At  Fleet  Numerical  Weather  Facility  (FNWF)  a 
modified  barotropic  model  is  in  use  as  no  known 
prognostic  model  has  proven  capable  of  consistently 
outperforming  the  FNWF  model.  A  great  deal  of  the 
success  displayed  by  this  model  comes  from  its  having 
been  tuned  (i.e.  adjusted  for  maximum  accuracy)  over 
a  period  of  several  years. 

The  Weather  Bureau  has  recently  instituted  a  numer¬ 
ical  model  based  on  the  hydrodynamical  equations  in 
their  unmodified  or  primitive  form.  With  the  speed  of 
the  third  generation  computers,  it  is  now  feasible  to 
integrate  the  equations  in  short  time  steps.  However,  all 
models  suffer  from  the  lack  of  observational  data. 
Among  existing  operational  centers,  differences  in 
initial  analyses  are  often  as  great  as  differences  in  24- 
hour  forecasts.  Without  a  correct  analysis,  any  forecast, 
no  matter  how  good  the  model,  is  going  to  be  in  'rror. 
A  plateau  has  seemed  to  be  reached  in  numerical  fore¬ 
casting  and  any  substantial  increase  in  accuracy  can 
only  be  attained  with  an  improvement  in  initial  data. 

The  greatest  increase  in  forecast  accuracy  since  1960 
was  achieved  recently  when  the  U.S.  Air  Force  imple¬ 
mented  its  Automated  Weather  Network  (AWN). 
Observations  are  collected  from  regional  teletype 
weather  circuits  terminating  in  computers  at  High 
Wycombe,  England  and  Fuchu,  Japan.  This  data  is  im¬ 
mediately  transmitted  by  high-speed  communication 
links  to  a  collection  center  at  Tinker  AFB,  Okla.  From 
there,  it  is  relayed  together  with  western  hemisphere 
data  at  approximately  6,000  teletype  error-free  words 
per  minute  to  FNWF  with  drops  also  at  the  Weather 
Bureau  at  Suitland,  Md.,  and  the  Air  Force  Global 
Weather  Central  at  Offutt  AFB,  Neb. 

Within  a  few  years,  this  initial  stimulus  was  followed 
by  the  establishment  of  similar  research  groups  in 
Sv'eden,  Elngland,  Japan,  and  Germany.  In  the  United 
States,  a  landmark  was  reached  in  1956  when  an  upper- 
air  numerical  forecast  map  was  disseminated  via  the 
national  weather  facsimile  network  to  replace  a  sub¬ 
jectively  prepared  one. 

It  is  natural  to  ask  what  makes  the  problem  of 
weather  prediction  so  unique  and  so  demanding  of 
human  wit  and  brute  machine  force. 

First,  the  atmosphere  exhibits  undulatory  and  vortex 
motions  on  a  tremendous  variety  of  length  scales.  These 
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range  from  millimeter-size  eddies,  spinning  along  a 
windswept  ground  surface,  to  the  thousand-mile-size 
vortices  or  cyclones  associated  with  major  storms.  A 
typical  hemispheric  weather  map  is  shown  in  £gure 
VIII-1.  All  of  these  motions,  small  or  large  in  scale, 
are  directly  or  indirectly  essential  parts  of  the  weather- 
producing  mechanism.  Thus,  many  observations  of 
variables  are  required  simply  to  describe  the  details  of 
the  current  weather. 

Even  assuming  that  the  overall  effect  of  very  small- 
scale  eddies  is  determined  by  events  on  a  larger  scale, 
it  is  conservatively  estimated  that  the  values  of  at  least 
five  variables  (pressure,  temperature,  humidity,  wind 
speed  and  direction)  would  be  required  at  5,000 
unifoi'mly  spaced  points  at  each  of  10  different  altitudes 
to  provide  a  sufficiently  accurate  description  of  the  state 
of  the  atmosphere  at  a  single  instant. 

Another  complicating  factor  is  the  solution  of  the 
equations.  The  form  of  the  hydrodynamical  equations 
is  such  that  the  instantaneous  local  time  derivative  of 
each  variable  can  be  expressed  in  terms  of  space  deriva¬ 
tives  of  variables  in  the  same  set.  Accordingly,  if  we 
can  observe  thr  initial  values  of  ell  variables  at  a  net¬ 
work  of  discrete  points  filling  the  entire  atmosphere, 
we  can  approximate  the  relevant  space  derivatives  by 
the  method  of  finite  differences,  and  can  compute  the 
initial  rate  of  change  of  each  variable  from  the  hydro- 
dynamical  equations.  Knowing  the  initial  value  and 
initial  rate  of  change  of  each  variable,  we  can  then 
extrapolate  its  value  over  a  vei  y  short  interval  of  time 
at  each  point  in  the  network.  Finally,  regard  this  very 
short  range  forecast  as  a  new  set  of  initial  data  and 
repeat  the  process.  Thus,  it  is  possible  to  build  up  a 
prediction  over  any  desired  period  of  time  as  a  series 
of  successive  forecasts  over  very  short  intervals  of  time. 

The  real  problem,  obviously,  is  to  carry  out  the  over 
1  billion  calculations  required  for  a  24-hour  forecast 
in  considerably  less  time  th.in  24  hours.  Herein  lies  the 
power  of  the  high-speed  computer. 

The  average  number  of  reports  received  per  opera¬ 


tional  run  is  as  follows: 

•Surface  pressure  (land)  4000 

Surface  pressure  (ship)  550 

f  'pper  sir  soundinas  510 

Bsiliyihermoprsphs  (60 -r  hour  collerlion  j^erind  .580 
.''es  surface  irmiwrslure  ).V-j  =  dsy  collrclion 

period)  7800 


Figures  ¥111-2  and  VIll— .1  illustrate  the  density  of 
surface  weather  and  upi>er  air  repsirts  received  for  3 
Novemlrer  1966. 

The  data  is  analyzed  at  Monterev  each  12  hours  on 
grids  of  4.000  points  I  Figure  VIII— 4,  soon  to  be  in¬ 


creased  to  16,000  points,  for  the  northern  hemisphere 
from  a  depth  of  1,500  feet  in  the  ocean  to  100,000  feet 
in  the  stratosphere.  Hemispheric  forecasts  are  made  out 
to  72  hours  for  upper-air  levels  and  48  hours  for  the 
surface.  These  forecasts  require  about  30  minutes  of 
computer  time  or  on  the  order  of  2  billion  computa¬ 
tions.  The  fields  are  transmitted  at  4,000  words  per 
minute  to  Fleet  Weather  Centrals  in  Guam;  Pearl 
Harbor;  Norfolk,  Va.,  and  Rota,  Spain.  It  is  planned  in 
,ae  future  to  shift  to  communication  satellite  channels 
at  up  40,000  words  per  minute.  At  the  Weather 
Centrals,  the  data  is  tailored  for  the  areas  of  responsi¬ 
bility  and  transmitted  to  fleet  usere.  The  Naval  En¬ 
vironmental  Data  Network  is  illustrated  in  figure 
VlII-5.  Data  tielines  (fig.  VIII-6.)  on  the  U.S.  east 
and  west  coasts  p'-ovide  environmental  products  to 
Naval  Weather  Service  Commead  units  and  other 
Government  agencies. 

The  growth  in  computer  speeds  from  the  beginning 
of  routine  numerical  weather  prediction  is  depicted  in 
figure  VIII— 7.  The  computational  mix  for  the  curve 
assumes  ten  adds  for  every  fixed  voint  multiply  in 
typical  meteorological  work.  The  figure  showl^  that 
speeds  underwent  a  steady  lo;: -rithmic  increese  during 
this  period. 

Although  the  dollar  co.  ‘  of  the  computer  systems 
used  in  meteorological  centers  has  increased  in  the  last 
10  years,  the  time  (and  consequently,  the  cost)  per 
operation  has  shown  a  remarkable  decrease.  This  may 
be  seen  in  figure  VIII-8  which  summarizes  the  time 
and  cost  of  five  million  typical  meteorol  :.'gical  com¬ 
putations.  Figure  v’lll-'  giv::?  a  nummary  of  the 
Control  Data  (CDC)  computers  in  operation  at  FNWF 
as  well  as  the  new  STAR  computer  scheduled  for  in¬ 
stallation  in  fiscal  year  1969.  Not  indicated  is  a  CDC 
8090  communication  computer  used  for  data  inter¬ 
change  with  Tinker  AFB  in  the  Automated  Weather 
Network. 

With  the  increase  in  computer  speeds  it  is  now 
possible  to  perform  numerical  integration  for  long 
periods  of  time  to  determine  the  atmospheric  circula¬ 
tions.  Such  computations  are  scientifically  important  I.- 
that  they  require  a  formulation  of  many  of  the  physical 
processes  occurring  in  the  atmoaphere.  These  processes 
include  the  effects  of  orography,  moisture,  turbulence 
and  radiation.  Integrations  have  been  carried  out  for 
several  hundred  days  and  have  given  realistic  r«uits  in 
correspondence  with  climatological  atatiatica.  However, 
numerical  forecasts  still  fail  to  show  any  realistic  skill 
after  2  to  .3  days. 

There  are  manv  processes  in  the  atmo^ihere  which 
are  poorly  understood.  They  include  release  of  latent 
heat  when  water  vapor  condentes  into  cloud  droplets. 
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Ficufc  \lll  -i  —  Htmupheric  foresails. 


I  absorption  of  the  sun's  radiant  heat  energy,  less  of  heat 
i  hy  radiation  and  heating  by  conduction  from  a  \»ariner 
j  land  or  sea  surface.  These  problems  have  to  he  solve<l 
f  before  longer  range  forecasts  are  meaningful, 
i  What  will  the  decade  ahead  bring?  It  is  iie.zardous  to 
S  forecast  the  course  of  science.  Clearly,  however,  it  will 

!  he  reqiiireci  to  bring  the  whol'*  earth’s  weather  unde- 

the  kind  of  round-the-world  observational  scrutiny  that 
is  going  to  l>e  required  for  operation  of  the  weather 
forecast  models  of  1077.  .A  tenfold  increase  will  l>r 
rc(]uire<l  in  nicteorologiral  coverage  hy  the  weather¬ 
station  network  of  the  late  1070’s. 


Satellites,  free-floating  balloons  hy  the  thousands, 
remote,  mannesi  and  unmanned  land  and  ocean  weather 
stations  these  and  other  tools  must  he  marshaled. 
Computers,  even  with  the  wisest  of  men  behind  them, 
cannot  lie  exjrrrted  to  make  accurate  2-week  weather 
prcslictions,  unless  they  are  fed  with  correct  weather 
information  on  an  hour-to-hour,  day-to-day  basis. 

Perhaps  von  Neumann  was  right  in  the  ..'Sessment 
that  he  made  in  the  mid- 1050’s  that  the  weather  prob¬ 
lem  is  the  greatest  challenge  for  the  application  of 
computer  science  to  Irencfil  mankind. 
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TIME  AND  COST  OF  5  MILLION  METEOROLOGICAL 

COMPUTATIONS 


Mr.  Hacke.  Are  you  using  LF  for  conununicaticns?  same  category  as  forecasting  the  future  of  technology 
Commander  Hamilton.  No,  they  go  on  facsimile  and  or  population,  but  you  have  giver  many  things  which 

tdetype  circuits  and  so  forth.  we  suspected.  The  laws  are  not  fully  understood  and  I 

Mr.  Smith.  How  ans  you  using  weather  satellites  for  **'•"*'  **  P*P*'  illustrated  the  importance  and 
{diotography  in  weather  forecasts?  reveals  where  understanding  is  inadequate,  and, 

Commander  Hamilton.  Tlfc  satellite  we  feel  is  ^^^hermore,  reveals  where  data  is  insufiBcient.  From 

basically  a  research  tool.  We/use  it,  for  example,  in  "tandpoint  perhaps  those  of  us  in  the  forecasting 

areas  of  no  data.  This  can  give  you  a  pretty  good  idea,  **“*‘’'“* 
for  example,  that  we  don’t  have  a  storm  located  quite 

right-in  Washington  they  digitise  this  data  or  locus  '‘‘T’  comments  or  quw- 

it,  in  other  word^  for  the  height  and  we  put  them  into  P‘P*‘  *"  relationship 

our  analyseo.  This  is  tho  only  operational  use  being  *  **  ' 

made  of  them.  It  is  valuable  for  pic  ting  storms  where  0»eRBECk.  I  feel  there  are  further  implications 

data  is  unavailable.  Other  than  th'  it  is  primarily  a  P“P**^  '^**P*®*  demographic 

research  tool.  The  satellite  is  very  interesting  and  predictions.  The  motivation  for  establishing  a  large 
intriguing  but  operationally  it  is  not  much  u&e.  Since  ®  collection  and  data  processing  system,  is  the  need 
thu  is  a  panel  of  long  range  forecasting,  I  should  men-  **•**  information.  It  has  many  operational 

tion  that,  my  specialty  is  numerical  work,  some  of  the  “**8.  In  the  more  complex  case  of  data  collection  for 
outlook  for  meteorology  in  the  future,  westher  control.  demographic  projection  from  a  census  every  10  years, 
I  am  no  expert  on  this— it  is  a  problem  similar  to  that  ^cr  improving  forecasting  methodology 

merticiied  *his  mon.ing,  of  putting  oil  on  the  water  to  **  direct  requirement  needs  for  population 

stop  evaporation.  Somebody  is  going  to  be  hurt— even  data.  We  can  conjecture  these  but  I  am  not  clear  what 
with  hurricanes— first,  there  is  so  much  about  the  mechanism  i»  to  make  this  kind  of  data  economi- 

atmosphere  that  we  don’t  understand,  or  even  why  a  ^  wonder  if  the  people  here  could  make 

hurricane  starts.  Why  is  it  transformed  from  a  weak  »“gg«t»on9  in  the  direction  of  what  would  be  economi- 

diatukbance  into  a  full-blown  hurricane?  A  few  years  “**^“*-  ^  conjectures  but  I  would  prefer 

ago  we  thought  there  had  to  be  ice  clouds  to  act  as  experts  do  this. 

nuclei  to  start  raindrops  then  we  realised  in  the  tropics  Mr.  Irwin.  Are  you  specifically  interested  in  getting 
this  doesn’t  happen.  We  don’t  clearly  understand  the  mote  census  data? 

rain-producing  mechanism.  Before  we  can  do  much  Dr.  Oberbeck.  I  am  saying  for  meteorological 
about  the  field  of  weather  control,  we  have  to  under-  science,  you  have  an  abundance  of  data  and  an  elabo- 
stand  the  atmosphere.  There  is  a  big  push  in  research  rate  mechanism  to  collect  it  on  c  hemispheric  scale, 

in  these  fields.  For  example,  the  rainmak'  rs — one  The  motivation  for  all  of  this  was  someone  needed 

fanner  may  want  rain  on  his  field,  hut  in  the  city  it  laps  information.  It  was  economically  useful  and  I  am  sure 
over,  and  may  disturb  the  businessmen  that  people  are  there  must  be  an  analogy  for  people  who  need  popula- 
not  buying  because  it  is  rainmg.  I  read,  the  other  day  tion  data.  I  was  conjecturing  with  Dr.  Eber  that  we 

when  I  was  in  Stockholm,  that  the  head  of  the  Swedish  have  the  technology  in  the  United  States  and  in  the 

Weather  Bureau  was  president  of  the  World  Meteor-  world  today  so  we  could,  for  example,  take  census  data 

ological  Organisation.  He  was  talking  about  the  once  a  week  with  satellites.  Have  population  informa- 

Russians  firing  a  can^  j  into  the  clouds  and  laughed  tion  collectors  out  continually  and  let  the  satellites  pick 

about  it.  He  thought  it  was  very  funny,  but  the  Ameri-  up  their  information.  I  wonder  if  your  office  has  toe 

can  scientists  are  taking  a  very  hard  look  at  dispersing  means  for  stimulating  the  use  of  this  data  and 

hail-producing  clouds.  This  would  have  a  tremendous  justifying  it  economically. 

economic  effect  on  the  farmers  in  Nebraska  if  they  Mr.  Irwin.  There  has  been  talk  of  a  census  cverv  5 
TOuld  do  this— hail  costs  them  millions  and  millions  of  y^rs  and  1  understand  the  House  has  passed  a  law  that 

dollars  damage  a  year.  This  would  be  profitable  to  ,i,ere  will  be  a  census  in  1975  and  there  was  consider- 

everybody  and  nobody  would  be  hurt.  debate  in  public  hearing.  1.  having  it  in  -lf^65. 

Major  Martino.  There  was  a  specific  reason  for  Whether  there  will  be  an  appropriation  when  the  time 

bringing  a  paper  on  weather  forecasting  in  this  meet-  comes  is  another  rtory.  The  feeling  in  the  field  is  that 

ing.  This  approach  fits  what  Dr.  Johnston  called  pre-  all  people  using  demographic  statistics  apparently  feel 

dictions.  That  is.  in  principal  we  think  we  know  all  the  10  years  is  too  long.  It  is  probably  the  pickup  in  the 

laws  that  govern  the  situation  and  it  doesn’t  fit  in  the  rate  of  technological  process  in  the  recent  years  and  a 
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greater  use  of  demographic  data.  You  may  not  be 
aware  that  every  month  the  Bureau  of  Census  has  a 
sample  household  survey  which  collects  important  data. 
The  design  is  fundamentally  to  produce  labor  force 
data  for  Denis’s  organization,  and  they  provide  most 

of  the  money  to  support  it— 35,000  householdti - 

Dr.  Johnston.  It’s  50,000  now. 

Mr.  Irwin.  Most  of  the  money  comes  from  the  Labor 
Department  in  various  months  of  the  year — March  is  a 
heavy  month,  also  October.  Certain  questions  are  put 
^  in  as  riders;  in  addition  to  asking  the  basic  labor  force 
data  in  March,  they  ask  questions  about  marital  status, 
children  they  have;  and  various  other  things  and  then 
we  do  a  collection  on  special  order.  The  outfit  in  Census 
1  that  works  up  the  sample  statistics  and  runs  the  office 
is  much  larger  than  the  population  division,  and  they 
are  located  in  a  couple  of  large  wings.  One  serious  prob¬ 
lem  is  to  get  more  census  data.  I  don’t  think  people  will 
hold  still  for  a  population  register.  I’m  not  sure  a 
population  register  is  a  good  idea  because  this  country 
moves  around  fast.  You’ve  got  to  generate  a  tre¬ 
mendous  amount  of  data.  There  has  been  a  discussion 
about  collecting  social  security  numbers  in  the  next 
c  ensus.  However,  it  has  not  been  decided  whether  social 
security  information  will  be  collected.  In  Philadelphia, 
for  example,  we  were  testing  procedures  for  complete 
coverage  o'  housing  population,  we’re  mailed  question¬ 
naires,  ond  later  we  knocked  on  every  door  to  be  sure 
thai  all  housing  units  in  the  area  were  on  our  mailing 
list.  About  10  days  later  we  knocked  on  the  door  again 
asking  for  population  information.  I  observed  that  the 
people  were  extremely  reluctant,  actively  antagonistic: 
“Go  away.”  There  :s  a  certain  human  equation  in 
getting  data  from  people.  I  don’t  know  if  this  is  the 
kind  of  thing  you  are  interested  in. 

Dr.  Oberbeck.  Let  me  amplify  what  I  had  in  mind. 

I  want  to  draw  an  analogy  between  collecting  demo¬ 
graphic  data  and  collecting  oceanographic  data.  In 
World  War  II,  1  was  in  the  submarine  service.  Just  as 
your  householders  complained  about  collecting  data 
from  them  so  the  people  in  the  submarine  service  col¬ 
lecting  bathythermic  data  griped  because  they  didn’t 
know  what  would  be  done  with  it;  and  it  was  an  inter¬ 
ference  with  their  daily  routine.  An  antidote  to  this 
kind  of  reluctance  is  finding  s<itneone  who  can  use  the 
data  collected  and  pay  for  this  valuable  information. 
An  improvement  for  this  whole  area  would  be  if  you 
could  induce  the  labor  force  to  say.  “We  really  have  to 
have  these  labor  force  projections.  We  will  pay  lor 
getting  the  data  and  get  it.” 

Dr.  Johnston.  One  possibility  is  if  we  should  get  a 
negative  inc  ime  tax  or  legislation  of  a  similar  type.  1 


would  be  in  favor  of  it  though  everyone  would  receive 
a  check  from  the  Government  which  would  be  returned 
together  with  more  than  you  ever  before  put  in  with 
you  income  tax  payment,  but  the  mechanism  of  putting 
each  member  of  society  on  a  per  capita  basis— some 
kind  of  a  valued  item  like  money — this  might  solve 
some  of  our  undercount.  It  would  probably  give  us  an 
overcount,  but  would  serve  as  a  possible  entry  justify¬ 
ing  the  collection  of  needed  information  without  quite 
giving  rise  to  the  hostility  and  suspicion  received 
knocking  on  th’-  same  door  twice.  The  situation  is 
structured  by  the  respondent  in  a  totally  fantastic  way. 
He  thinkj  the  FBI  is  on  him  or  somebody  else  is  after 
him  and  you  can’t  undo  this  structure.  You  are  caugh^, 
as  Dick  Irwin  says,  in  a  pathology  to  where  you  may 
get  no  information  or  lies.  I  think  one  of  the  long-range 
schemes  in  the  UN  has  been  for  a  worldwide  sampling 
procedure  but  it  is  not  in  the  planning  stage.  What  is 
the  stage  before  you  plan?  You  evade,  you  conjec¬ 
ture — a  kind  of  sampling  device  you  have  in  weather 
but  on  a  worldwide  scale,  getting  at  people’s  conditions. 
A  register,  as  Dick  Irwin  says,  is  probably  too  much 
although  it  seems  to  function  well  in  countries  like 
Sweden  and  Denmark.  It  is  likely  not  to  be  feasible  in 
most  places,  but  a  sampling  technique  on  a  v  orldwide 
scale  is  not  entirely  inconceivable.  If  we  can  process 
data  for  50,000  households,  publish  data  in  about  2 
weeks  and  if  we  had  the  real  technology  in  operation 
we  could  probably  do  it  faster,  there’s  no  reason  we 
can’t  do  it  with  other  households. 

Major  Martino.  I’d  like  to  raise  another  possibility. 
Is  there  any  reason  why  the  census  people  could  not 
take  into  account  the  same  privacy  rules  and  make  use 
of  data  obtained  from  banks,  credit  bureaus,  and  so  on, 
where  in  general  the  social  security  number  is  avail¬ 
able? 

Mr.  Irwin.  You  could  do  it  with  social  security. 
We’ve  made  many  tests  trying  to  improve  the  enumera¬ 
tion — I  should  say  in  defense  of  ourselves — I  men¬ 
tioned  the  undcrcount — 1  think  the  Bureau  has  accom¬ 
plished  a  great  dce.l  trying  to  count  everybody  and  I 
want  to  mention-- if  we  had  the  social  security  number 
it  would  work — but  in  lest  after  test  matching  lists 
against  lists,  the  information  is  not  complete  enough. 
You  can’t  do  it  and  it  hasn’t  worked.  You  should  be 
aware  that  the  Bureau  is  making  a  dramatic  change  in 
this  next  census— the  decision  had  definitely  been  made 
to  do  a  mail-out  mail-back  census  tc  all  urban  areas  i.i 
the  Nation,  covering  80  percent  of  the  population.  Jim 
Hacke  inentionetl  one  of  the  ba.»ic  problems  that  came 
under  my  observation  in  the  Philadelphia  pretest. 
First,  we  found  in  the  l'I60  census  that  self-enumera- 
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tion  the  best.  The  way  it  worked — there  was  an 
amioer^tor  checked  on  every  house,  and  every  fourth 
house  in  the  sample  household  an  enumeration  check* 
a*boz*type  questionnaire  was  left  and  was  mailed  in. 
If  there  were  any  problems  she  talked  to  them  again. 
We  used  self  •enumeration  on  the  sample  data;  on  100- 
.  percent  data  there  was  also  a  100-percent  pickup  so  it 
was  not  exactly  a  self-enumeration.  We  are  asking  for 
a  positive  schedule  this  time,  there’s  little  doubt  from 
it,  and  the  schedule  is  tough.  I  am  perfectly  confident — 
I  am  speaking  for  myself  not  the  Bureau — there  are  a 
lot  of  people  who  are  extremely  experienced  working 
in  this  complicated  schedule.  We  are  going  to  get 
extiemeiy  good  information  from  people  with  high 
educational  attainment.  But  I  observed  a  few  ghetto 
interviews  of  people  with  an  eighth  grade  education 
and  the  household  situation  was  such  that  you  couldn’t 
concentrate.  For  instance  in  an  academic  atmosphere 
similar  to  yours,  I  can  use  complicated  sentence  struc¬ 
ture  and  you  people  pretty  well  understand  what  I  am 
saying.  However,  in  the  ghetto  kids  are  running  around 
wishing  you  were  somewhere  else.  T*'e  fine  print  goes 
flying  out  the  window,  and  these  are  a  few  of  the 
problems  in  collecting  data.  1  think  seif-enumeration 
is  the  right  way  to  do  it. 

Dr.  Linstone.  The  more  I  hear  about  this,  the  more 
puxxled  I  am,  considering,  this  country  is  the  moat 
advanced  technologically.  I  know  in  Germany  before 
Hitler,  every  person  was  required  to  register  with  the 
police.  It  was  standard  procedure  to  rer^ster  with  the 
police  every  time  you  moved.  One  of  the  reasons  for  the 
low  crime  rate  in  Japan  is  the  police  have  an  up-to-date 
dossier  on  all  individuals,  which  is  standard  operating 
procedure.  If  these  countries  can  do  it,  I  don’t  under¬ 
stand  why  we  esn’t. 

Mr.  Irwiw.  First,  it  is  impossible  psychologically  in 
this  country.  Since  the  beginning  of  this  whole  organi- 
ution,  you  don’t  do  that  kind  of  thing  and  that’s  why 
the  Census  Bureau  is  reluctant  to  get  a  social  security 
number  and  I,  as  an  American  citizen,  am  opposed  to  it. 
I  think  one  of  the  strengths  of  this  country  is  the 
freedom. 

Dr.  Eacii.  I  was  in  Germany — which  is  about  the  size 
of  Texas — for  many  years  and  the  main  thing — if  you 
look  at  the  size  of  your  country— I  imsgine  the  reason 
you  couldn’t  do  it  is  the  bigr.ess  of  the  country. 

Mr.  Irwin.  I  don't  know  much  sbout  foreign  coun¬ 
tries  but  suspect  we  sr«  more  mchilr  than  anybody. 
For  one  thing,  we’ve  so  many  more  automobiles. 

Dr.  Linstone.  1  don’t  understand  the  philosophic 
objection — which  sounds  like  it  is  anti-American.  I 
don't  tee  why  everyone — nobody  seems  to  object  to  the 


idea  that  you  are  part  of  the  system,  however,  you 
mentioned  your  own  opposition  to  it.  Does  it  inhibit 
your  movements? 

Mr.  Kacxe.  1  can  answer  that  in  a  semiphilosophical 
or  theological  way  by  saying  Americana  suspect  that 
every  human  being,  if  you  have  enough  on  him,  would 
be  sent  to  jail,  and  you  are  not  aent  to  jail  unless  you 
are  caught.  It  goes  back  to  the  froi/ier  psychology. 
You  don’t  ask  a  man  where  he  has  been.  He  makes  his 
way  on  what  he  is  currently  doing. 

Mr.  Irwin.  In  the  West  nobody  asks  who  your  rela¬ 
tives  are  or  where  you  cap"e  from.  Nobody  cares.  In 
Galifumia,  they  could  care  less  where  you  are  from. 

Dr.  Oberbeck.  Another  aspect  of  this  in  the  reverse 
sense  of  the  investigation  in  the  ghetto  is  a  personal 
reaction  to  giving  data  to  people  whose  education  you 
suspect  may  be  less  than  yours.  As  an  example,  I  find 
locally  getting  license  plates  for  your  automobile,  the 
people  rendering  this  service  aren’t  on  the  edrcational 
level  to  understand  you  want  this  done  quickly.  They 
let  you  stand  around  all  morning;  they’re  doing  their 
job  and  could  care  less  about  your  time.  I  believe  the 
population  data  could  be  done  by  mail,  and  wouldn’t 
be  nearly  as  objectionable  as  having  to  present  myself, 
say,  before  a  police  officer  to  tell  him  who  I  am  and 
answer  questions  of  the  kind  necessary  for  a  population 
survey. 

Major  Martino  I  would  like  to  expand  on  your 
earlier  question,  Tom.  You  pointed  out  that  we  have 
mammoth  weather  service  because  somebody  decided 
he  wanted  it  enough  to  pay  for  it,  your  original  ques¬ 
tion  was:  What  would  it  take  to  convince  somebody  he 
should  be  willing  to  pay  for  data  and  in  this  case  not 
only  the  monetary  cost  but  the  psychological  cost — 
what  would  we  have  to  do  to  propagandize  people  so 
they  will  go  after  this  census  question?  What  is  neces¬ 
sary  to  make  it  economically  and  socially  feasible  to  do 
what  we  know  could  be  done,  with  census  takers?  I 
would  like  to  expand  that  to  take  in  the  interests  of  the 
other  people  as  well.  What  needs  to  be  and  can  be  done, 
economically  or  socially  feasible,  to  gather  the  kinds  of 
data  we  think  we  need  on  technology,  population  anti 
on  all  the  other  things  that  most  of  us  seem  to  be 
interested  in  forecasting?  Does  anyone  wish  ?o  address 
or  comment  on  it? 

Dr.  Obfrbcck.  Let  me  amplify  your  amplii^cation 
with  a  question.  In  what  tense  is  the  Gallup  poll  eco¬ 
nomically  productive? 

Dr.  Slafkoskt.  Newspapers  buy  it. 

Dr.  Oberbeck.  It’s  economically  feasible,  I  guess  and 
has  betm  going  on  for  t  long  time. 


Major  Martino  When  somebody  is  running  a  politi¬ 
cal  campaign  nowadays,  he  takes  a  sample  poll.  To 
reduce  the  cost  a  member  of  one  party  will  combine 
with  a  member  of  another  party,  whom  he  ib  not  run¬ 
ning  against  and  is  a  candidate  for  a  different  office, 
to  jointly  pay  the  coat  of  the  poll.  It  is  an  accepted 
method  of  doing  business  in  politics.  Here  is  somebody 
who  wants  it  badly  enough  to  pay. 

Mr.  Smith.  The  military  has  not  been  unwilling  to 
pay  money  for  forecasts.  The  Air  Force  Project  Fore¬ 
cast  is  not  cheap,  never  was.  CNM  is  about  to  invest — 
in  the  millions  of  dollars — to  get  a  Navy  technological 
forecast.  Apparently  the  concept  of  the  value  of  a  tech¬ 
nological  forecast  has  been  well  establis'ied,  at  least 
throughout  the  military.  Companies  represented  by 
lA/ckheed  and  others  are  investing  considerable 
amounts  of  money  in  forecasting.  I  think  we  are  ad¬ 
dressing  the  wrong  question.  It  isn’t  what  it  takes  to 
sell  its  worth,  it  is  more  a  question  cf  implementation 
than  a  question  of  need.  It  apparently  has  been  accepted, 
perhaps  we  should  address  the  question  of  what  is  the 
psychological  impact  or  how  do  we  overcome  that. 
In  the  case  of  the  census,  it  appears  to  be  a  stumbling 
block  and  possibly  in  other  services  it  is  also  psycho¬ 
logical. 

Major  Erb.  I  wonder  if  part  of  the  problem  is  that 
some  of  the  disciplines — in  the  area  of  international 
relations  and  political  science — don’t  have  the  proper 
methodology  in  which  to  apply  the  data.  There  are 
many  people — we’ve  come  in  contact  with  groups  at 
Yale  and  at  USC — trying  to  get  contracts  to  collect 
data.  One  reason  they  want  to  collect  data  is  to  figure 
what  types  of  models  you  could  build  to  describe  the 
world  if  you  had  the  data.  I  think  if  the  time  came 
where  there  were  sufficiently  reliable  models  available 
to  explain  parts  of  the  world  people  would  be  interested 
and  willing  to  pay  for  the  data.  Right  now  we  are  at 
the  stage  where  the  data  are  not  good  enough  but  the 
techniques  used  for  handling  the  data  aren’t  there 
either. 

Major  Martino.  This  sounds  like  a  vicious  ciicle. 
Regarding  initiation  of  operation  research  studio,  with¬ 
out  data,  you  can’t  build  a  model,  and  without  s  model, 
you  don't  know  what  data  to  gather. 

Dr.  SiArKOSKY.  I  think  we  are  asking  the  wrong 
questions.  Tom  Oberheck  is)  asking  about  the  Callup 
polls.  Dick  Irwin  is  talking  about  census  people  recog¬ 
nising  that  people  have  the  data  and  the  problem  of 
extricating  it  from  them  because  they  are  reluctant. 
But  here  it  isn't  that  kind  of  problem.  The  real  problem 
is  we  don't  know  enough  yet.  The  people  who  have  been 
working  in  this  area  aren’t  kidding  themselves  about 


how  good  they  are.  They  recognize  we  are  at  the 
beginning  of  this  whole  process  and  it  is  not  a  question 
of  how  much  money  is  spent  but  that  there  aren’t 
enough  people  who  have  mastered  the  technique.  It  is 
not  only  a  question  of  not  enough  people,  also  that 
techniques  have  not  been  perfected — if  we  were  able  to 
perfect  them,  then  the  data  would  be  relatively  easy  to 
get.  This  is  my  feeling.  I  don’t  know  if  we  could  ever 
compare  ourselves  with  the  census,  the  Gallup  poll  or 
anything  else.  It’s  a  much  more  difficult  process  that 
we  are  trying  to  get. 

Major  Erb.  Part  of  the  problem,  is  that  in  some  areas 
you  don’t  know  what  to  collect.  If  you  are  trying  to 
understand  the  international  system  as  a  system,  what 
are  you  going  to  look  for?  Are  you  going  to  look  for 
an  area,  population,  or  are  you  looking  for  messages 
transferred  between  various  divisions?  We  don’t  have 
a  good  enough  system  to  determine  what  we  want  to 
collect  even  if  we  had  the  wherewithal  to  collect  h. 

Mr.  Hacke  One  comment — essentially  until  you 
know  what  you  are  looking  for,  you  can’t  see  it,  so  to 
speak.  The  other  is,  I  wonder  if  the  question  has  not 
been  phrased  backwards:  About  how  can  we  get  people 
to  invest  more  money.  In  census  taking,  it  isn’t  the 
question.  It’s  the  optimum  level  of  effort  for  census 
taking,  in  general  and  detail,  and  how  you  arrive  at 
that  optimum. 

Major  Martino.  I  think  we  ell  suspect  we  are  short 
of  the  optimum. 

Dr.  Slafkosky.  It  seems  we  have  at  least  three  kinds 
of  information  or  insights  we  are  looking  for.  One 
insight  the  Major  pointed  out,  namely,  where  are  the 
conflicts  going  to  be,  what  is  going  to  be  the  nature  of 
the  conflicts  which  depends  where  they  are  going  to 
take  place.  The  second  question  is:  Given  that  we  know 
who  is  going  to  be  doing  it  how  do  we  come  to  gripe 
with  that  nation  or  that  group  of  people?  The  thing 
people  have  been  calling  the  social  technology,  which 
I  am  always  questioning,  because  it  sounds  too  much 
like  a  mechanical  thing,  it  is  really  relevant.  Ute  third: 
Let’s  assume  we  know  who  is  going  to  be  involved. 
What  are  the  best  instruments  from  the  technological 
standpoint?  I  am  talking  about  hardware,  that  will  do 
the  job  most  efficaciously  and  moat  efficaciously  doesn't 
necessarily  mean  what  so  many  of  our  congressmen 
now  seem  to  feel,  “bomb  them  off  the  map"  or  "over¬ 
come  them  somehow."  I  think  most  efficaciously  m^ns 
how  do  you  impose  vour  will — maybe  the  word  “will" 
is  wrong — how  do  you  get  the  people  to  work  with  you 
as  a  nation  or  a  group  of  nations  for  the  betterment  of 
the  world.  I  may  sound  a  little  idealist  but  unless  we 
more  toward  an  idealistic  goal,  this  is  going  to  be  a 
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nt  noe  and  will  end  with  the  deitruction  of  everybody. 
I  nay  be  only  talking  about  three  kinds  of  things; 
dien  may  be  more.  But  these  are  the  obvious. 

Major  ManiNO.  You  raised  a  point  we  have  to  look 
at  As  an  aside,  it  is  not  clear  that  we  are  doing  as  well 
by^droj^ing  bombs  as  we  might  do  by  dropping  money, 
connteileit  ration  cards  or  whatever  most  easily  dis- 
nqita  their  society.  But  you  are  right. 

Dr.  Slavkosky.  The  British  did  this  in  Malaya, 
they  didn’t  drop  money  but  boy,  did  they  use  h. 

Major  Swierr.  We  might  try  dropping  Sears  Roebuck 
catalogs.  It  seems  to  be  an  implicit  consensus  that  social 
problems  underly  technological  forecasting.  It  diould 
be  noted  that  social  sciences  are  dealing  with  two 
classei  of  problems.  One  is  fact  and  the  other  norma* 
tive,  ’’What  is”  and  ’’what  should  be;”  the  d>ould-be 
side  is  the  leading  bias  for  intentions.  A  decisionmaker 
looks  at  his  perception  of  presoit  and  future  reality 
and  looks  at  bis  perception  of  what  ought  to  be  now 
and  in  the  future.  Between  these  is  the  momentum  for 
his  action.  Then  he  starts  looking  for  means,  he  has 
some  that  he  prefers;  and  then  looks  at  his  capabilities 
to  see  which  can  be  used.  This  question  of  what  ought 
to  be  is  the  same  question  of  utility  and  drops  back  into 
the  underdeveloped  area  oi  axiology  in  philosophy  and 
psychology.  If  wo  would  unlock  anything  in  that  area, 
we  could  unlock  a  considerable  chain  of  interdependent 
proUenu.  I  think  as  far  as  the  implementation  of  what 
we  can  do  about  our  problems  is  concerned,  as  a  sug* 
geation  we  might  use  something  like  the  cohort  theory 
from  demography  and  see  if  we  couldn’t  study  what 
vahiea,  what  views  of  reality,  are  going  into  the  minds 
of  these  cohorts.  We  have  seen  the  Red  Guards  in 
Communist  China.  We  know  that  15  to  20  years  ago  the 
young  children  were  being  taught  to  kill  the  ’’dirty 
capitalist  flies”  and  shoot  the  “dirty  imperialirt  rats.” 
They  were  weaned  from  milk  onto  revolution.  Revolu¬ 
tion  and  violence  became  a  positive  value  there.  But 
now  the  people  that  sowed  the  wind  are  reaping  a 
whirfwind.  This  type  of  approach,  I  think,  might  be 
fruitful  to  all  of  our  problems.  The  Red  Guard,  at 
least,  might  have  been  predicted. 

Major  Martino.  I  would  like  tc  ask  Dr.  Oalkey 
about  his  oonunent.  The  first  reaction  is  to  say,  “let's 
do  more  social  science,”  and  the  second  reaction  is  to 
suggest  maybe  we  don't  need  to  know,  or  understand, 
if  are  make  use  of  the  insights  we  have  acquired.  People 
have  been  looking  at  people  for  a  long  time  and  doing 
pntly  weU.  Would  you  care  to  expand  on  your  earlier 
comments? 

Dr.  DaUCKY.  If  you  look  over  technology  and  science 
it  is  pntty  dear  thst  technology  as  a  whole  has  ted 


science  rather  than  vice  versa.  Most  scientists  are  in¬ 
clined  to  think  the  reverse — they  have  been  brought 
up  thinking  the  reverse.  As  a  matter  of  fact,  science 
has  followed  technology,  and  for  a  very  good  reason. 

The  steam  engine  came  before  thermodynamics.  The 
telephone  came  before  information  theory.  In  short, 
one  of  the  motivatioiu  for  scientific  research  is  the 
demonstration  of  a  coutroUed  capability.  In  the  case 
of  social  sciences,  there  is  a  very  good  chance  they  will  s 
follow  the  same  route  if  we  can  demonstrate  it  is  possi¬ 
ble  to  be  effective  in  influencing  social  systems.  That 
would  be  a  tremendous  impetus  for  looking  into  the 
reason  why.  At  the  moment  we  could  make  more  use 
of  what  has  come  to  be  known  as  social  inventions, 
rather  than  more  social  sciences.  A  great  social  inven¬ 
tion  is  money.  The  question  is  with  regard  to  problems  I 
of  international  conflict.  Gm  we  find  s  device  which  | 
will  enable  international  transactions  to  take  place  | 
witliout  conflict.  I  don’t  know  of  such  a  device.  William  | 
James  made  a  point  of  the  moral  equivalent  of  war. 
Nobody  has  a  moral  equivalent  yet.  I  don’t  tee  why  such 
a  thing  shouldn’t  be  invented  without  the  intervention 
of  social  science. 

Mr.  Hacke.  Before  I  make  my  point,  1  note  Dr. 
Dalkey’s  example  that  technology  came  before  science, 
from  the  19th  century,  and  I  wonder  if  that  is  still 
true.  There  may  be  something  in  terms  of  what  kind 
of  social  invention  might  lead  to  a  moral  equivalent  of 
war.  I  don’t  intend  to  be  facetious  but  have  you  noticed 
how  much  of  the  organquition  of  service  industries  and 
merchandising  has  beeh  toward  control  of  depersonal¬ 
ising  the  relationship  where  you  go  to  a  gas  station, 
ask  for  credit  and  the  man  says,  “It’s  not  the  station’s 
policy  to  give  credit.”  What  he  means  is  he  is  not 
giving  credit,  but  puts  it  in  an  impersonal  form  and 
much  of  the  merchandising  drive  has  been  toward 
depersopaliting.  That’s  all  right  io  a  study  of  conflicts 
in  the  restaurant  situation  which  come  from  the  fact 
that  waitresses  are  lower  on  the  ladder  than  cooks,  give 
orders  to  the  cooks  and  invented  that  wonderful  gadget 
you  see  in  some  restaurants — whirligig  where  they  put 
the  orders.  The  reason  that  gadget  ia  there  is  so  the 
waitros  doesn’t  tell  the  cook  or  hand  him  the  order. 

She  puts  it  on  and  he  takes  it  off  impersonally.  Maybe 
we  can  find  a  whirligig  where  people  can  put  their 
aggreesive  internal  onal  siguaU  into  an  impersonal 
mechanism  and  get  them  picked  up  on  the  other  side 
of  the  outfit.  The  point  I  originally  wan'ed  to  make 
was:  Social  scientists  neein  to  be  overly  apologetic 
about  their  lack  of  hardnoaed  methodology  and  par¬ 
ticularly  in  psychofdiyaks. 
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Colonel  Vaughn.  I’m  not  trying  to  bring  the  conver* 
sation  away  from  Olympian  levels,  but  I  want  to  ask 
this  question  because  it  may  possibly  lead  back  to  the 
Olympian  levels.  I  was  quite  intrigued  in  Commander 
Hamilton’s  talk  about  the  Russians  and  Chinese  coop¬ 
erating  with  us  in  this  particular  endeavor.  This,  of 
course,  is  out  of  the  usual  pattern  and  when  any^ing  is 
out  of  the  usual  pattern,  you  look  twice,  because  there 
may  be  clues  to  otlier  interesting  tangents  and  we  might 
go  through  several  tangent  actions.  Would  you  care  to 
comment  why,  particularly  the  Chinese? 

Commander  Hamilton.  The  only  reason  is  the 
weather  in  Western  Europe.  The  weather  over  the 
United  States  will  end  up  in  Europe  and  eventually 
affect  Russia  and  China,  so  it’s  a  mutual  cooperation — 
mutual  benefit,  you  might  uy.  Beside  weather  data, 
which  is  the  main  interchange,  there  is  a  hotline  from 
Washington  to  Moscow  and  we  transmit  our  weather 
satellite  broadcasts  to  them  and  they  in  turn  transmit 
them  to  us.  I  don’t  think  anybody  knows  exactly  why 
but  it  must  be  the  old  philosophy;  If  it  wasn’t  doing 
them  any  good,  they  wouldn’t  he  doing  it.  It  must  be 
based  on  this  because  of  the  weather  as  I  showed  you 
,  the  whole  hemispheric  map — now  going  to  the  whole 
globe  you  can  get  a  complete  weather  picture.  The 
larger  the  area  the  better  your  forecast. 

Colonel  Vaughn.  A  very  significant  th'ng  is  the  fact 
not  only  do  we  think  so,  but  apparently  they  do  also, 
otherwise,  why  would  they  do  it?  It  would  be  most 
interesting  to  know  particularly  in  the  last  year  with 
the  Red  Guard  eruptions,  many  things  have  happened 
that  look  like  sheer  insanity  yet  they  treat  it  as  an  every 
day  occurrence. 

Conunander  Hamilton.  Take  any  data  last  week  and 
compare  it  to  1,  2  years  ago — it  won’t  vary,  they  may 
have  their  Red  Guards,  revolution,  and  everything  else, 
but  it  hasn’t  affected  their  weather  communications. 

Dr.  Linstone.  I  was  curious  as  to  what  this  coopera¬ 
tion  actually  consists  since  we  have  no  diplomatic 
relations. 

Commander  Hamilton.  Through  the  auspices  of  the 
World  Meteorological  Organisation,  the  nations 
throughout  use  a  common  weather  code.  In  other  words, 
five  digit  teletype  groups.  There  are  local  variations 
which  are  authorised  by  the  ^orld  Meteorological 
Organisation.  We  all  know  the  other’s  variations  which 
is  a  fairly  standard  code.  When  we  receive  this  infor¬ 
mation  at  6,000-words-a-minute  from  Tinker  Air  Force 
Bate,  it  goes  into  our  ADP,  the  data  processing,  and  is 
never  touched  by  hand.  It  is  edited,  reformatted,  and 
run  through  the  complete  data  processing  based  on 
Mandard  codea,  and  cornea  out  as  raw  data.  This  may 


not  be  your  point  but  they  do  cooperate  in  common 
code.  For  example,  you  could  report  at  different  heights 
but  everybody  has  agreed  to  report  at  a  certain  stand¬ 
ard  reporting  level.  Tbia  type  of  cooperation  makes 
it  easier. 

Dr.  Slafkosky.  Let  me  rephrase  the  question.  What 
you  said  was  they  are  cooperating — agreeing  to  stand¬ 
ard  forms  and  terminology;  and — ^in  the  sense  that 
they  are  receiving  our  data  and  analysea.  How  are  they 
cooperating  by  giving  na  information?  The  inference 
1  got  from  Colonel  Vaughn’s  statement  was  they  were 
in  fact  providing  some  information  heartily. 

Commander  II-xMiLTON.  In  the  last  3  years  there  have 
been  three  World  Meteorological  Centers :  One  in  Aus¬ 
tralia,  Moscow,  and  Washington  which  are  common 
links.  I  mentioned  the  hotline  from  Washington  to 
Moscow.  All  their  teletype  daU  besides  this  automated 
weather  work  was  set  up  by  the  Air  Force  akhongh 
the  Weather  Bureau  is  also  using  it.  There  is  a  standard 
teletype  weather  broadcast  and  radio  facsimile  even  in 
the  Mediterranean. 

Major  Stephenson.  Are  you  saying  the  cooperation 
exists  not  because  of  a  need  for  cooperating  between 
the  two  groups,  but  because  vrithin  their  society  they 
must  transmit  weather  data  back  and  forth  and  we 
ride  the  circuit  to  get  it  by  intercept? 

Commander  Hamilton.  We  do  take  it  by  intercept 
but  there  are  standard  teletype  circuits  into  the  central 
collecting  centers  as  in  Moscow  and  then  are  trans¬ 
mitted  to  the  stations  in  Europe  and  the  United  States. 

Major  Martino.  You  say  they  voluntarily  provide 
weather  information? 

Commander  Hamilton.  Yes, 

Mr.  Hacke.  They  provide  data  in  the  Soviet  Union 
and  China? 

Conunander  Hamilton.  Yes,  1  might  say  aero-aero 
and  1200  are  the  standard  times  for  observation  and 
they  cooperate.  It  would  be  useieas  if  they  had  staggered 
times  because  you  couldn’t  draw  a  weather  map  but 
all  are  taking  their  observations  at  the  aame  time. 

Major  Martino.  So  they  cooperate  by  using  the  aame 
format  and  ways  of  making  rnmnuremanta  aa  every¬ 
body  elae? 

Commander  Hamilton.  Right.  They  t^  us  and 
there’s  no  problem. 

Major  Eaa.  1  was  going  to  comment  on  something 
that  interested  me  when  I  was  in  Korea  a  few  years 
bark.  Wh«i  filing  for  a  flight  we  received  a  weather 
briefing  and  in  every  military  woUher  station  you  aae 
a  aeriea  of  yellow  pages  with  12-hour  reports.  The 
military  weather  station  in  Kor««  always  had  the  latest 
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WMther  report  from  Pyong  Yang,  North  Korea,  which 
waa  picked  up  on  their  normal  teletype  sequence. 

Major  SwiTT.  This  question  of  science  versus  tech* 
nology  is  another  chicken-and'-egg  problem.  The  idea 
of  social  technology  in  the  blind  is  what  worries  me. 
When  you  don’t  know  the  likely  results  ot  your  input, 
you  get  die  same  thing  Mr.  Hacke  was  discussing,  un¬ 
foreseen  impacts  whose  resuk  may  or  may  not  be  what 
yon  had  m  mind.  This  is  my  point,  “Let’s  find  out  a 
little  more  about  what  we  are  doing  before  jumping 
into  a  big  program.”  In  the  jargon  of  the  social 
sciences,  this  is  called  social  tinkering,  which  I  like 
better  than  social  technology. 

Dr.  LotSTONE.  You  mentioned  money  as  a  social  tool 
and  this  tinkering  question  come  in — in  the  sense  that 
the  aid  money  we  are  expending  may  be  tinkering.  It 
may  create  more  problems  than  it  solves.  Unless  we  are 
careful,  and  this  is  an  important  point  as  in  meteorol¬ 
ogy  with  the  ocean  currents  and  air  currents,  we  don’t 
know  what  we  need  to  do  about  the  interactions,  there¬ 
fore,  tinkering  may  have  an  adverse  effect  on  the  whole 
system.  The  same  is  true  here.  It  is  quite  dangerous  to 
look  for  tools  m  technology  that  will  do  someAing — in 
a  sense  yon  are  tinkering  on  a  public  scale,  as  in 
cBmatobgy. 

Dr.  Dalket.  In  this  circumstance,  the  caricature 
would  be  somebody  saying  to  James  Watt,  “Look,  have 
yon  thought  about  the  social  consequences  of  your  in- 
vauEion,”  and  his  answer  probably  would  have  been, 
"No.”  Aasnme  someone  perceptive  would  have  been 
abk  to  poiirt  out  the  horrors  of  the  early  industrial 
revolution.  The  picture  would  have  been  pretty  Biblical 
and  if  I  had  been  Janies  Watt  and  someone  had  been 
abb  to  convince  inc  they  thought  a  revolution  would 
be  the  outcome  of  it,  I  might  have  been  inclined  to 
forget  it.  I  doubt  if  the  judgment  of  history  is  that  it 
sfould  have  been  better  if  he  had  forgotten  it.  That’s 
my  caricature. 

Dr.  OuniECK.  Following  that  line  of  discussion,  the 
Wright  brothers,  silling  in  judgment  of  modem  air¬ 
ports,  might  have  done  the  same. 

Major  MAKTi’to.  I  think  the  term  “social  tinkering” 
has  buried  in  fc  some  value  judgments  which  is  a  bit 
sneaky. 

Major  SwETT.  The  value  judgment  of  it  is  the  ides 
of  making  a  small  move  to  see  what  happens;  and  is 
essentially  a  pragmatic  value.  .\s  far  as  social  tinkering 
we  have  been  doing  it  ad  infinitum,  from  time  out  of 
mind.  Wc  can  look  at  the  experiment  we  call  America 
or  the  experiment  we  call  Russia  which  are  really  the 
James  Watt  type  social  technology  approaches:  If  we 
don't  learn  from  th«n,  we  are  irrestponsible  to  say  the 


least.  Let’s  look  at  the  results  the  people  before  ua 
received — who  had  to  make  the  decisions.  They  had  to 
tinker  and  invent;  let’s  see  the  results  of  their  attempt. 

Dr.  S1.AFKO8KY.  I  am  not  sure  the  mass  of  people — 
we  don’t  like  to  call  it  social  adencee  to  nuike  predic¬ 
tions  as  to  what  would  follow  from  Watt’s  invention, 
hut  they  would  have  foieaeen  the  things  you  arc  talking 
about  were  going  to  happen.  The  fact  that  these  hap¬ 
pened,  I  am  convinced,  is  the  result  of  certain  happen¬ 
ings  and  events  and  people  took  hold  and  did  certain 
things  with  it.  It  isn’t  something  you  or  anyone  could 
have  foreseen.  I  don’t  know  that  we  can  use  this  type  of 
quasihistorical  argument.  I  purposely  call  it  “quaai” 
because  I  don’t  think  it’s  really  one — in  looking  at 
possible  implications.  You  are  not  saying,  “Look  at  all 
the  implications  before  you  leap.’’  You  are  saying,  “if 
you  want  to  accomplish  certain  objectives,  they  are 
more  than  technological,”  and  I’m  talking  about  hard¬ 
ware,  ways  of  getting  them  if  you  take  the  hardware 
bit,  you  are  not  sure  of  getting  there.  Examine  it  thor¬ 
oughly  ;  look  at  all  the  other  avenues  of  approach  you 
might  use  and  try  to  see  which  ones  will  marry  up  the 
beat.  Am  I  understanding  you  properly? 

Major  SwETT.  Yes.  There  is  another  jargon  term 
from  my  own  discipline.  In  international  relations  the 
approach  is  essentially  pragmatic  like  this:  “Dear 
Prince,  Yours  truly,  Machiavclli."  Michiavclli  has  had 
a  bad  press  but  was  essentially  prag-aatic  and  says  “if 
you  want  to  kick  over  a  monarchy,  here’s  what  you  do. 
If  you  want  to  kick  over  a  republic,  do  this.  Here’s 
what  you  do  not  do  and  why.”  That’s  the  value  we 
could  contribute  to  tech  forecasting  and  planning. 

Dr.  SiAFKOSKY.  He  pointed  out  the  implication  of 
these  things  orally  and  every  other  way. 

Dr.  JoHNSTOtf.  I  suggest  all  this  sounds  Utopian  and 
that  computer  stimulation  would  afford  a  substitute  for 
social  tinkering.  We  had  a  bad  experience,  trying  to 
look  at  soci^y  as  an  experiment  because  we  never  had 
control  over  the  variables  with  respect  to  past  action. 
This  is  the  problem  of  the  historian  versus  the  scientist. 
Our  problem  is  we  sre  tied  up  with  historical  processes 
that  never  repeat  themselves  and  have  the  same  prob¬ 
lem  when  we  extrapolate.  We  would  like  to  have  a 
typical  period  of  prosperous  times  with  full  employ- 
merit  and  extrapolate,  but  it  can’t  be  found.  The  closer 
you  look  at  history,  the  more  you  see:  You  had  reces¬ 
sion,  then  a  recovery,  another  abnormal  perio<l,  and 
then  a  baby  boom,  everything  is  abnormal  in  that 
respect,  it  may  be  that  computer  stimulation  ultimately 
would  be  the  shemstivc. 

Major  Swrrr.  This  is  a  good  point.  Such  work  ha* 
gone  on  at  Northwestern  with  Dr.  Cuetxkow  and  been 
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very  stimulating  in  this  area.  In  other  words,  you  don’t 
have  to  wait  for  the  earth  model  to  run  itself  out,  if 
you  can  come  up  with  decent  approximations  to  speed 
the  time  factor.  A  run  was  made  several  years  ago  with 
four  exchange  students  taking  part  as  decisionmakers 
in  the  simulated  countries.  Dr.  Levine,  an  anthropolo¬ 
gist,  asked  to  predict  in  writing  how  they  would  run 
their  countries  based  on  his  knowledge  of  their  ethnic 
background.  He  predicted  nine  points  of  what  each 
would  do.  The  first  thing  he  asked  was:  “What  tribes 
are  they  from?”  He  predicted  from  thct  basis  and  cor¬ 
related  exactly  wit!-,  what  they  did  in  subsequent  runs 
of  the  simulation  and  how  they  went  about  it.  One  was 
from  a  tribe  that  had  been  victorious  in  battle  for 
nuny  years.  Most  of  their  holdings  were  acquired  by 
clubbing  somebody  which  was  precisely  how  he  oper¬ 
ated.  I  checked  Dr.  Levine's  predictions  with  current 
newspaper  rqrarts.  The  same  tribes  are  still  doing  the 
same  things. 

Dr.  SiAFKOSKY  I  couldn’t  agree  with  you  more  when 
you  are  dealing  with  individual  incidents,  repetition 
of  which  will  never  occur  because  the  act,  the  time, 
and  everything  else  is  different.  We  may  learn  some¬ 
thing  from  this  field  but  we  will  never  learn  it  in  the 
same  sense  that  we  are  supposed  to  learn  from  science, 
especially  the  mathematical  sciences  where  we  have 
complete  control  of  the  assumptions,  the  postulates, 
and  the  rest  is  simply  logic.  Even  the  physical  sciences 
where  we  are  dealing  with  inert  matter  and  can  repeat 
things.  However,  I  want  to  bring  to  your  attention  a 
statement  1  have  heard  several  times  in  the  last  few 
days:  Mainly,  that  sciences  have  laws.  The  physical 
sciences  do  not  have  laws  any  more  so  than  the  social 
sciences.  If  we  look  at  the  physical  sciences  you  will 
say,  “Yes,  they  have  laws.  These  laws  are  by  and  large 
patterns  that  account  for  the  phenomena  available  to 
us.”  These  laws  change  and  have  changed.  For  over 
two  thousand  years  they  have  been  changing  as  well 
as  the  last  10  years,  and  will  continue  to  change. 

Major  Stephenson.  One  reason  why  the  physical 
sciences  have  been  successful  is  they  have  been  careful 
with  questions  they  addrosed.  Many  questions  within 
the  physical  sciences  were  no  less  complex  than  the 
current  questions  within  social  sciences  except  th<Me 
in  the  physical  sciences  people  know  better  than  to  ask 
those  queationa. 


Dr.  Slafkosky.  The  physical  scientist  has  it  easy 
and  can  cut  off  what  he  wants  to  look  at  more  easily 
than  the  social  scientist  dealing  with  an  individual 
person  who  is  theoretically  unique  from  anybody  else, 
whereas  the  only  uniqueness  about  an  atom  of  one 
kind  as  opposed  to  another  of  the  same  kind  is  q>atial. 

Commander  Hamilton.  You  will  agree  that  the 
whole  atmosphere  is  a  big  laboratory  and  you  can’t  cut 
it  down  but  I  agree  this  is  true  in  most  physical  sciences 
and  you  have  a  control  laboratory. 

Dr.  Dalkey.  Tlierc  is  a  great  deal  of  wisdom  to  be 
gained  from  using  meteorology.  In  your  illustration,  if 
the  physical  s<  -entists  had  taken  the  problem  of  man¬ 
aging  the  atmosphere  in  the  19th  century,  they  would 
have  been  about  in  the  same  position  as  the  social  scien¬ 
tists  are  who  are  trying  to  manage  a  large  society,  and 
physical  scientists  just  refused  to  accept  that  as  their 
problem.  I  don’t  see  any  reason  why  social  scientists 
couldn’t  refuse  to  accept  this  tremendously  mobile 
problem  and  try  something  smaller.  However,  there  is 
one  small  counter  implication  and  that’s  the  case  of 
the  Soviet  Union,  as  I  understand  it,  economists  in  the 
United  States  have  taken  a  great  deal  of  satisfaction 
with  the  shift  toward  western  types  of  economic  man¬ 
agement  going  on  in  the  Soviet  Union.  The  conclusion 
is  the  great  experiment  of  controlled  economy  has 
failed.  The  United  States  is  doing  a  good  deal  better 
economically  than  the  Soviet  Union.  However,  look  at 
it  from  another  point,  it  is  anything  but  a  failure. 
Consider  the  incredible  problem  they  took  on  them¬ 
selves:  to  manage  that  economy.  Maybe  they  didn’t  do 
it  perfectly  or  optimally,  or  as  well  as  the  United  States 
but,  they  did  it.  It  makes  you  think  of  the  story  of 
Johnson  and  Boswell.  They  went  to  a  vaudeville  show 
where  there  was,  among  other  things,  a  dog  act,  the 
dog  prancing  around  on  his  hind  legs  and  after  the 
show  Boswell  said,  “Wasn’t  it  wonderful  they  way  that 
dog  walked  around  on  his  hind  legs?”  Johnson  said, 
“That’s  not  the  nay  to  put  it.  It  wasn’t  wonderful  the 
way  he  did  it;  it  was  wonderful  he  did  it  at  all.”  You 
can  make  this  same  comment  with  regard  to  the  man¬ 
agement  of  an  economy.  The  fact  they  did  it  at  all  is 
something.  I  think  we  can  say,  out  of  that  comes  the 
lesson  something  as  large  as  an  economy  can  be 
mastered.  It's  not  beyond  the  capabilities  of  economic 
science. 


One  of  the  reasons  for  having  a  technological  forecast  it  to  use  it  in  making  pdans 
for  allocation  of  R.  <t  D.  resources.  Mr.  Celron  and  Mr.  5mith  describe  a  proced-ure 
currently  being  used  by  the  Navy  to  allocate  a  portion  of  its  R.  &  I),  resources  which 
incorporates  a  technological  forecast  as  part  of  the  information  going  into  the  alloca¬ 
tion  procjfeaaL 

— Editor 


165 


I 

t 


I 

I 

1 


TECHNOLOGICAL  FORECASTING  IN  MILITARY  PLANNING 

Mar.'In  J.  Catron,  Haadquart^n,  Naval  Maitrlal  Laboratory 
Donald  F.  Smith,  Naval  Ship  Rnaoarch  and  Oovo/opmont  Cantor 

Forecast;  It  «  possible  to  achieve  by  1980  fuel-cell  powerplants  having  a  lifespan  of 
2  years. 

Forecast:  In  the  same  time  period,  marine  gas  turbines  operating  with  inlet  tempera¬ 
tures  of  2,300°  F.  are  also  achievable. 

Forecast:  A  50-percent  reduction  in  the  volume  of  marine  steam  powerplants  is  tech- 
nically  feasible  by  1980. 


These  forecasts,  and  others,  stem  from  the  work  of 
a  team  of  U.S.  Navy  scientists  and  engineers  at  the 
Annapolis  Division  of  the  Naval  Ship  Research  and 
Development  Center,  where  they  are  using  technology 
itself  to  detail  future  technological  developments. 
Technological  forecasting  approaches  are  helping  the 
U.S.  military  services  decide  which  avenues  of  research 
and  development  offer  the  largest  capability  payoffs  in 
terms  of  military  missions.  For  example,  if  the  military 
mission  is  to  seek  ou‘  and  <lestroy  other  submarines,  it 
might  be  asked  whether  it  is  better  to  do  research  on 
quieting  the  engines  of  a  submarine  or  on  increasing 
the  power.  Clearly,  if  money  and  time  were  unlimited, 
both  pieces  of  research  could  go  on  in  parallel,  and 
similarly,  all  possible  research  could  be  pursued  cn  all 
subjects.  But  with  military  systems  becoming  more  and 
more  complex,  planners  are  having  to  make  increas¬ 
ingly  more  difficult  choices.  They  need  a  better  tool 
than  “gut  feeling.”  The  new  art  or  science  of  tech¬ 
nological  forecasting  is  sure  to  become  the  new  mixlus 
vivendi. 

The  most  obvious  ann  jKrhaps  easiest  methiKl  to  use 
III  the  attempt  to  foretell  the  future  is  to  assume  that 
past  t-ends  will  continue.  Most  intuitive  forecasts  of 
progress  arc  probably  basevl  on  this  rclativelv  inaccu¬ 
rate  rxtrapolat:  >n  technique.  ITie  less  obvious  but  more 
scientific  techniques  are  based  on  quantitative  cxtrajKi- 
lat  .ons  (hut  arr  rrprt^urihlr  hr  knt>wlcd^ral>lr  workers 
in  t.He  field. 

(brer  the  perioil  of  life  of  a  technical  development, 
progress  as  a  fmvetion  of  time  most  often  assumes  an 
S-shaped  curve.  The  growth  of  a  technology  is  similar 
to  the  growth  of  bacteria.  .Ad’^ance  is  slow  during  the 


initiation  period.  Then  an  accelerated  growth  occurs 
exponentially.  At  later  stages,  approaching  and  during 
maturity  the  effects  of  limitations,  due  perhaps  to 
theoretical  limits,  cause  a  deceleration,  and  the  progress 
curve  levels  off. 

The  U.S.  Navy  must  perform  certain  wartime  mis¬ 
sions  such  as  target  detection  and  destruction,  strategic 
deterrence  (Polaris),  and  amphibious  operations.  The 
Mission.'Technology  organization  chan  (fig.  IX-1) 
shows  how  the  design  characteristics  of  vehicles  that 
would  be  needed  to  accomplish  the  given  missions  arc 
derived.  While  each  mission  would  call  for  vehicles 
with  different  characteristics  optimized,  all  vehicles 
would  have  many  functions  in  common,  such  as  detec¬ 
tion,  propulsion,  offense,  defense,  etc, 

A  forecast  ran  be  made  for  each  of  these  functions. 
For  instance,  propulsion  forcirrsts  would  be  in  terms  of 
specific  weight  of  the  propulsion  system,  its  reliability, 
noise,  and  whatever  other  parameters  are  considered 
significant  for  the  mission.  It  should  be  understood  thet 
the  I'.S.  .Navy  u.ses  the  same  shioa  for  mar.y  lvp«»  of 
missions  and  therefore  tradetiffs  arc  required.  The  sub¬ 
system  segments  examined  would  be  the  propuLsion 
transmission  svstem,  energy  converter,  thrust  producer, 
and  whaleyer  other  segments  are  needed. 

In  this  fashion,  we  ran  work  our  wav  down  the  chart, 
eventually  going  into  anv  degree  of  detail  we  wish. 
This  information  is  iircd  for  verv  prartiea!  ends.  For 
example,  here's  part  of  the  forera,irt  b  .  gas  turbines. 

(.^as  turbines  that  have  high  jKiwer  and  light  weight 
and  that  ran  operate  in  a  sah  sea-air  environment, 
eoulil  well  l>e  the  p^wrrplants  that  make  it  possible  for 
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U8  to  have  advanced  hydrofoil  craft,  suhtender  hydro- 
ahimmera  and  other  high  speed  (100  kts.  and  better) 
naval  vessels. 

Marine  gas  turbines  have  a  tremendous  potential  for 
development.  The  possibilities  for  high-power,  light¬ 
weight,  com^^  act  powerplants  are  unmatched  in  any 
other  type  of  unit.  Powerplants  with  these  character¬ 
istics  are  particularly  vital  for  powering  new-concept 
vessels  such  as  hydrofoils  and  aircushion  craft.  See 
illustrations  IX-1,  IX-2,  and  IX-3. 

A  rapid  growth  in  the  power  capabilities  of  gas  tur¬ 
bine  units  has  occurred.  Engines  as  large  as  43,000 
horsepower  have  been  built,  and  units  exceeding  50,000 
horsepower  are  projected.  This  growth  trend  will  prob¬ 
ably  continue  but  at  a  lesser  rate  as  limitations  of 
mechanical,  thermal,  and  ducting  size  factors  are  ap¬ 
proached.  However,  much  larger  power  outputs  will 
be  obtained  by  using  multiple  gas  generators  to  drive 
a  single  power  turbine.  Power  outputs  as  high  as 
150,000  horsepower  have  already  been  attained  by  this 
method. 

The  history  of  development  in  areas  of  weight,  vol¬ 
ume,  and  fuel  consumption  for  the  simple  cycle  gas 
turbine  are  shown  in  figure  IX— 2,  IX-3,  and  IX-4.  The 
engine  components,  namely  the  compressor,  combi  -  ■ 
tion  chamber,  and  turbine,  were  investigated  to  deter¬ 
mine  their  development  trends.  As  shown  in  figure 
IX-S  potential  improvements  in  compressor,  com¬ 
bustion,  and  turbine  efiiciency  are  slight.  Any  future 
improvements  extending  those  already  made  will  bring 
about  little  increase  in  efficiency;  consequently,  work 
to  improve  the  efficiency  of  these  components  will  have 
an  insignificant  effect  on  the  future  engine  perform¬ 
ances. 

Compressor  pressure  ratio  has  increased  significantly 
during  recent  years,  but  any  further  increase  will  be 
small.  Because  of  improvements  in  blade  loading,  com¬ 
pressors  are  now  designed  to  an  optimum  pressure 
ratio  determined  by  turbine  inlet  temperature.  Blade 
loading,  which  has  enabled  engines  to  obtain  higher 
pressures  with  fewer  stages,  appears  to  be  approaching 
a  practical  limit. 

The  addition  of  more  heat  energy  within  the  same 
basic  engine  configuration  has  been  the  major  con¬ 
tributing  factor  to  the  recent  and  future  improvement 
in  the  engine  characteristic  parameters.  Therefore,  the 
parameter  that  will  have  a  significant  effect  on  the 
engine  is  the  turbine  inlet  temperature.  Tlie  increase  of 
inlet  temperature  is  shown  in  figure  IX-6.  Extrapola¬ 
tion  of  the  curve  to  temperatures  in  excess  of  3,000°  F 
is  based  on  laboratory  tests  in  which  operating  tem¬ 
peratures  as  high  as  4,000°  F  have  been  achieved. 


It  can  be  seen  when  comparing  the  %ure8  that  the 
specific  weight  of  the  simple  gas  turbine,  volume,  and 
fuel-consumption  improvements  parallel  that  of  the 
turbine  inlet  temperature. 

Additional  savings  in  weight  and  volume  of  gas  tur¬ 
bines  can  be  expected  by  using  improved  materials  and 
better  structural  designs  and  blade-cooling  techniques. 
These  practices  have  been  applied  to  increase  inlet 
temperatures  to  today’s  figures,  but  major  increases  in 
this  area  are  still  possible. 

Thus,  from  a  practical  point  of  view,  research  and 
development  effort  should  be  concentrated  in  the  fol¬ 
lowing  areas  on  the  basis  of  the  high  payoff  possibili¬ 
ties  for  improvement  of  the  gas  turbine  for  marine 
purposes: 

•  Cooling  of  turbine  blades  and  other  components 
in  high  temperature  ambients.  This  will  allow 
higher  turbine  inlet  temperatures  that  will,  in  turn, 
lead  to  significantly  improved  performance  levels. 

•  New  materials  and  protective  coatings  for  these 
high  ambient-temperature  components.  This  will 
increase  life  expectancy  by  increasing  resistance 
to  high-teraperature  oxidation  and  sulfidation. 
An  increased  resistance  to  thermal  fatigue  and 
creep  is  also  required. 

•  Improved  materials,  designs,  and  fabrication 
techniques  for  regenerative  gas  turbines  to  re¬ 
duce  their  cost,  weight,  and  bulk. 

f  Further  application  and  adaptation  of  aircraft 
gas  turbines  and  technologies  to  marine  service. 

Technical  areas  where  relatively  low  payoffs  for  the 
expended  effort  could  be  expected  include: 

•  Attempts  to  improve  efficiency  of  combustion, 
compressor,  and  turbine.  These  efficiencies  in  the 
best  existing  units  are  about  as  high  as  can  reas¬ 
onably  be  obtained. 

•  Attempts  to  increase  significantly  the  blade-load¬ 
ing  or  compressor-pressure  ratio  unless  accom¬ 
panied  by  major  design  changes. 

We  can  match  technologies  against  the  needs  of  the 
U.S.  Navy.  The  planning  matrix  shown  in  figure  IX-7, 
based  on  hypothetical  vlues,  shows  that  in  the  gas- 
turbine  area,  there  are  still  major  problems  to  be 
solved  with  respect  to  small  boats,  integrated  Fleet 
propulsion,  integrated  ship  auxiliaries,  ship-noise  con¬ 
trol,  and  surface  effects  ships.  The  matrix  also  shows 
the  functional  areas  where  gas  turbines  are  applied. 

Three  factors  used  by  the  U.S.  Department  of  De¬ 
fense  to  evaluate  systems  programs  are  military  utility, 
technical  feasibility,  and  financial  acceptability.  These 
factors  are  also  important  when  planners  evaluate  re- 
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srsrch  and  development.  However,  it  is  neceaaary  to 
quantile  these  factors  so  that  they  may  be  compared 
for  different  Research  and  Development  progranu. 

Appraisal  she^  IX-l  addresses  the  problems  of 
military  utility.  Military  utility  with  espect  to  develop- 
ment  atr>oephere  is  a  measure  of  li.  &  D.  work  in  terms 
of  its  usefulness  in  meeting  U.o.  Navy's  General  Oper¬ 
ational  Requirements  (GOK).  To  be  usef.il,  hardware 
or  information  must  provide  a  new  or  improved  capa¬ 
bility  in  the  shortest  possible  time  after  its  need  is 
recognized.  Thus,  military  utility  is  made  up  of  three 
interdependent  criteria:  value  to  naval  warfare,  re¬ 
sponsiveness,  and  timeliness.  In  this  condensed  version, 
we  will  consider  value  to  naval  warfare. 

This  criterion  considers  the  exterit  of  the  contribu¬ 
tion  of  a  task  area  objective  (TAO),  a  unit  of  work, 
in  terms  of  its  inherent  value  as  well  as  its  military 
operational  value.  The  importance  of  a  task  is  measured 
by  its  relative  impact  on  any  individual  naval  warfare 
category  as  well  as  the  number  of  categories  receiving 
a  contribution  from  the  task  objective.  This  is  done  by 
multiplying  the  assigned  value  of  the  warfare  category 
by  the  impact  value  of  the  contribution  to  arrive  at  a 
value  for  each  individual  category.  The  sum  of  these 
values  will  determine  the  value  of  the  task  area  objec¬ 
tive  to  the  operator’s  guidance.  /Vote;  The  figures  of 
merit,  or  point  values  assigned  to  each  naval  warfare 
category  (col.  H  are  dummy  figure tlscy  were  as¬ 
signed  for  this  example  only.  The  actual  total  number 
of  points  assigned  these  29  naval  categories  are  equal 
to  100,  and  they  are  assigned  for  test  purposes  on  the 
basis  of  the  importance  of  each  of  these  categories  in 
the  1975—80  time  frame  since  this  is  when  most  of  our 
current  exploratory  development  »  ork  will  find  its  way 
into  the  Fleet.  Ilic  operational  users  provider!  the  test 
figures  based  on  the  present  world  situation  and  their 
estimates  of  the  most  probable  future  situations. 

When  the  warfare  area  specialist  fillet!  in  column  2, 
the  impact  of  the  task  area  objective  contributions,  he 
considered  the  d««:riptors  at  the  bottom  of  the  page 
(.Scale  of  IVfinitionsi .  In  some  rases  the  four  descrip¬ 
tors  do  not  adequately  describe  the  contribution;  in 
thoee  cates  he  interpolates  between  these  niurtbers. 

The  iredibiittr  of  the  ratings  of  (ecimical  fcatibility 
sad  the  probability  of  suectsa  incrr«te  if  they  are  rated 
by  penonnei  irbo  ksve  the  oeerssarr  technical  cxptsr- 
lisr  and  oonpeSencr.  as  they  car;  best  judge  these  fac¬ 
tors  on  the  basis  of  the  ability  and  experience  of  the 
inditidttfib  (tud  or  organizations  carryinf  on  the  cte- 
velopftw:r>i  efforts  under  considefation. 

The  lop  half  of  appraisal  -dieei  IX— 2  aoUcits  the 
opinion  ol  the  technical  specialist  regarding  the  [wob- 


ability  of  achieving  the  total  task  area  objective  that  is 
being  undertaken.  It  considers  whether  the  task  could 
be  successfully  sccomplished  from  a  scientific  and 
technical  feasibility  point  of  view.  Technical  risk  also 
takes  into  consideration  the  degree  of  confidence  or 
prediction  that  the  remaining  portion  of  the  total  ob¬ 
jective  can  be  attained.  The  degree  of  confidence  or 
prediction  that  the  remaining  portion  of  the  total  task 
objective  can  be  attained  usually  assesses  the  factors 
of  the  present  stste-of-the-art,  either  implicit  or  explicit 
This  technical  appraisal  is  naturally  based  on  technical 
forecasts  and  includes  time  fsetors  and  rwource  levels. 

Therefore,  the  technical  specialist  checks  the  box  that 
best  describes  his  opinion  regarding  the  task  area 
objective  being  evaluated,  as  well  as  the  numlwr  of 
differe:  t  concurrent  approaches  being  taken  which  are 
also  a  measure  of  probability  of  success. 

The  area  called  “sacred  cow?”  and  “who  says?”  was 
also  considered  in  what  we  call  the  “management  en¬ 
vironment.”  This  section  solicits  opinions  on  the  ac¬ 
ceptability  of  the  effort  in  the  management  structure. 
Here,  the  evaluator  is  asked  to  give  what  he  believes  to 
be  the  Washington  environment  considerations  con¬ 
cerning  this  effort,  and  he  checks  the  applicable  box. 

The  bottom  of  appraisal  sheet  IX-2  is  then  analyzed. 
The  total  program  is  calculated  by  value,  expected 
value,  and  desirability  index  for  three  funding  levels, 
by  the  computer.  The  inputs  for  military  utility  come 
from  Appraisal  Sheet  IX-l. 

For  example:  Suppose  the  proposed  task  area  objec¬ 
tive  (TAO).  or  R.  &  D.  effort,  is  to  devise  a  system 
able  '.o  detect  submerge*!  submarines  a  given  distance 
away  from  a  sensor,  sav  20  miles.  W'e  shall  consider 
the  criterion  “Value  to  Naval  Warfaie.”  Of  the  29 
naval  general  operational  requirements  shown  in  col¬ 
umn  1  of  appraisal  sheet  IX-!,  the  TAO  would  he  of 
value  and  contribute  only  to  five  GOR’s:  airborne 
.\SW,  surface  ASW',  submarine  ASW,  undersea  ASW. 
and  .ASW  ancillary  suppsirt. 

W'ith  re-sjvect  to  airtiome  ASW,  the  sner-est  of  the 
R.  a  n.  venture  in  this  hyprithetica!  example  is  con- 
siderei!  a  revolutionary  etlm.«ion  of  capabilities,  and 
is  a,  corded  0.7  point.  .At  the  .same  lime,  airtwme  ASW 
is  taki  to  contribute  5  nut  of  the  l()0  units  assigned  to 
ail  the  (X)R’a.  Thus,  the  value  of  the  T.AO  to  naval 
warfare  with  respect  to  airborne  ASW  is  0.7xfi~T.S. 
The  other  categories  can  he  similarly  evaluated  for 
their  conlribuliona.  and  the  total  value  of  this  T.AO  to 
naval  warfare  is  summed  at  11.'^  as  shown  cm  the 
appraiMl  aheet. 

For  our  caiculation  of  the  Prohahility  of  Sueexsa 
i  P.)  in  meeting  the  TAO.  we  use  the  probability  chart 
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4.  TOTAL  VALUE  TO  NAVAL  WARFARE 


11.4 


Sciue  nf  Definitions  for  "Impact  of  Task  Contribution"  (Column  2): 

Points  -  Pescrlptors 

1.0  Creation  of  radically  new  mission  concepts  (meets  overriding  critical  need) 
.7  Revolutionary  extension  of  capabilities 

.4  Incremental  or  marginal  inprovement  of  capabilities 

.2  Increase  in  econoiny 


Apphahal  Sheet  IX-1. 
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Appraisal  Summary 

No.  of  GOE’s  5 
Value  (V)  11.4 

Probability  of  Success  (pg)  0.9375 _ 

Expected  Value  (EV)  11.4  x  0.9375  =  10.7 

Optimum  Funding  $2  million _ 

Desirability  Index  (D)  5-35 _ 
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Table  IX-1. 
Tab'jlation  of  P, 
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ahown  on  table  IX-l.  In  this  charty  n  is  the  number  of 
concurrent  approaches  used  to  accomplish  the  TAO, 
and  C  is  a  number  arbitrarily  assigned  to  the  chances 
of  succeeding  in  a  given  approa'di.  We  use: 

80  -  100%  dumce  of  '^access:  C  =  0.8 
30  -  80%  chance  of  success:  C  =  0.5 
0  -  30%  chance  of  success:  C  =  0.2 

We  assume  that  all  approaches  n  have  the  same 
chance  of  succesc,  and  therefore  the  same  value  of  C. 
If  each  n  were  to  have  a  different  C,  a  more  involved 
calculation  would  have  been  nwessary. 

The  number  assigned  to  the  probability  of  one 
approach  failing  is  then  (1  —  C) . 

The  number  assigned  to  the  probability  of  all 
approaches  failing  is  (1  —  C)*. 

Further,  if  we  assume  that  at  least  one  of  the  ap¬ 
proaches  taken  will  succeed,  then  the  number  assigned 
to  the  probability  of  success  is  1—  (1  —  C)". 

This  ffgure  for  P,  is  filled  in  on  appraisal  sheet  No.  2 
under  the  Probability  of  Success  column. 

Example:  On  an  appraisal  ::heet  IX— 2,  we  might  have 
had  four  approaches  (n  —  4)  with  a  30-80-percent 
chance  of  meetmg  TAO  (C  =  0.5).  Then  the  number 
corresponding  to  the  probability  of  success  is  0.93750 
or  93.75  percent.  From  our  previous  example  we  calcu¬ 
lated  the  total  vnlue  of  a  given  TAO  to  be  11.4.  There¬ 
fore,  the  expected  value  is  11.4  X  0.9375  =  10.7. 

The  preceding  has  been  a  discussion  of  concurrent 
approaches.  If  the  task  area  were  made  up  of  phased  or 
sequential  operadons,  these  probabilities  would  be 
handled  in  a  different  manner. 

Hum  funding  levels  are  ittiliied  in  the  concur- 
current  approach:  the  actual/optimum,  maximum, 
minimum. 

The  actual/optimum  consists  of  the  latest  approved 
fiacal  data.  For  each  subsequent  year,  funds  are  entered 


based  on  what  is  estimated  as  necessary  to  achieve  the 
completion  date  if  the  task  area  is  supported  at  an 
optimum  rate.  An  optimum  rate  is  one  which  permits 
aggressive  prosecution  using  orderly  developmental 
procedures — not  a  crash  program. 

The  maximum  consists  of  what  could  effectively  be 
expended  in  advancing  the  task  area  completion  date. 
Maximum  funding  is  the  upper  limit  in  which  unlimited 
resources  are  assigned  in  order  to  accelerate  the  ac¬ 
complishment  of  a  task  area. 

The  minimum  consists  of  what  could  be  effectively 
utilized  to  maintain  continuity  of  effort  and  some 
progress  toward  fulfilling  the  task  area  objective. 
Minimum  funding  is  the  threshold  limit  below  which 
it  would  not  be  feasible  to  continue  further  efforts  in 
the  task  area. 


Desirability  Index  (D) 


The  simplified  formula  is: 

Value  ( V)  X  Prob.  of  success  (P,)  =  Expected  Value 

{EV) 

Expected  Value  (EV) 

Funding  Level  (C) 

These  techniques  are  k  .ended,  not  to  yield  decisions, 
but  rather  to  furnish  information  which  will  facilitate 
making  decisions.  Indeed  these  techniques  are  merely 
thinking  structures  to  force  methodical,  meticulous 
analysis.  The  data  plus  the  analysis  only  gave  us  in¬ 
formation.  It  takes  this  inforn.  Uion  plus  judgment  to 
render  the  decisions. 

There  are  certainly  technical  problems  in  making 
forecasts  even  if  the  methodology  is  firmly  established. 
One  of  these  areas  is  in  selecting  the  parameters  that 
are  to  be  forecast.  This,  incidentally,  is  the  first  step 
one  uses  to  prepare  a  technological  forecast  for  any 
area  of  interest. 

The  experience  of  such  experts  in  the  field  as  Ralph 
Lenz,  Jr.,  of  the  U.S.  Air  Force,  and  Prof.  James  Bright 
of  Harvard  University,  points  up  the  benefit  of  choosing 
param^ers  that  are  can  be  considered  independent 
o'  any  specific  technology  and  not  constrained  by  man¬ 
generated  rules  or  regulations.  The  approach  used  at 
the  Annapolis  Division  of  the  Naval  Ship  Research 
and  Development  Center  is  to  have  appropriate  tech¬ 
nical  experts  choose  parameters  characteristic  of  a 
functional  cafiability,  for  example,  pounds/horsepower 
for  energy  converters,  and  which  are  suflBciently 
descHptive  to  afford  a  scientific  assesnnent  of  specific 
techniques,  applicable  to  the  systems  under  considera¬ 
tion.  ! ,  '  < 

To  estabUshi  the  future  possibUities  of  an  energy 
converter, for  a  marine  propulsion  system,  one  must 
have  forecasts  of  at  least  the  following  parameters — 
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specific  weight,  specific  volume,  efficiency,  and  total 
power  capacity  per  unit.  These  predictions  or  forecasts 
are  associated  with  marine  energy  converters  in 
general.  By  summing  up  the  optimums  of  each  ca¬ 
pability,  we  arrive  at  an  idealized  energy  converter. 
However,  it  must  be  realized  that  a  unit  incorporating 
the  optimum  value  of  each  of  the  forecasted  capabilities 
will  probably  never  be  built  because  some  of  the 
capabilities  arrived  at  are  often  contradictory.  The 
lightest  unit  is  generally  not  the  highest  power  unit,  as 
an  example.  However,  the  value  of  the  analysis  is  that 
these  fimctional  capabilities  do  represent  a  art  of 
potentially  achievable  goals  for  development  programs 
if  one  is  willing  to  invest  sufficient  money. 

When  choosing  the  parameters  to  be  forecast  for 
specific  techniques,  planners  must  be  certain  to  select 
those  parameters  that  are  necessary  to  discriminate 
between  alternative  techniques.  With  respect  to  the 
example  of  energy  converters  for  a  ship  propulsion 
system,  if  a  selection  is  to  be  made  on  other  than  a 
subjective  basis,  forecasts  of  the  following  parameters 
must  be  made:  specific  weight,  specific  volume, 
efficiency,  reliability,  maintainability,  ease  of  automa¬ 
tion,  noise  and  cost.  Moreover,  if  the  analysis  is  to  be 
meaningful,  each  parameter  must  be  related  to  each 
other  for  each  technique.  It  would  certainly  be  faulty 
to  select  a  given  specific  technique,  for  example,  diesel 
engines,  as  the  best  powerplant  on  the  basis  of  efficiency 
and  ease  of  automation  without,  at  the  same  time, 
accepting  the  fact  that  among  other  detriments  the 
diesels  are  not  the  lightest  or  quietest  units. 

Once  the  planners  select  their  parameters  and 
circumscribe  the  areas  for  data  gathering,  they  must 
face  the.  problem  of  what  constitutes  valid  data.  The 
researcher  gathering  the  data  must  be  sufficiently 
knowledgeable  to  reject  fluke  data  points  but  open- 
minded  enough  to  accept  valid  points  perhaps  beyond 
his  personal  expectations.  The  U.S.  Navy’s  approach  to 
this  problem  is  to  use  engineers  trained  in  the  technical 
area  (electrical  engineers  for  electric  drives,  mechan¬ 
ical  engineers  for  steam  and  diesel),  but  not  personnel 
considered  expert,  to  do  the  literature  search  and  plot 
the  historic  data. 

.1 

To  derive  the  functional  capability  curves,  Ve- 
searchers  examines  the  record  setters  and  experimental 
units  in  order  to  provide  the  best  that  can  be  accom¬ 
plished  for  any  given  functiop.  On  the  other  hand,  'for 
the  specific  technique  curves,  only  data  from  opera¬ 
tional  equipment  are  used  so  that  the  curves  reflect  only 
practical  capabilities.  Each  data  point  is  throughly 
documented  and  background  material  filed.  These  files 
are  invaluable  when  curves  are  reviewed  by  the  experts. 


because  many  arguments  occur  when  data  points  do 
not  agree  with  subjective  opiaions. 

The  technical  library  is  invariably  the  starting  point 
in  the  search  for  supporting  data.  Early  data  are 
obtained  from  classic  texts,  which  also  indicate  the  life 
span  of  the  technology.  Technical  reports  usually  are 
the  source  of  recent  and  recent-past  data,  particularly 
reports  of  ship  trials  for  Navy  projects.  Obviously,  the 
mcst  time-consuming  element  in  generating  a  tedi- 
nological  forecast  is  in  this  data  gathering,  and  it  can 
often  be  costly  as  well.  In  some  instances,  library  re¬ 
search  is  not  adequate,  for  example,  where  reliability 
data  are  needed.  Here,  no  good  data  exist  in  recogniz¬ 
able  form  and  must  be  accumulated  at  the  source,  that 
is,  from  ship  records,  shipyards,  and  manufacturers, 
and  reduced  before  it  is  useful  for  forecasting. 

For  the  extrapolated  portions  of  the  specific  tech¬ 
nique  curves,  which  are  in  fact  the  forecasts,  several 
constraints  are  imposed.  First,  wherever  possible,  a. 
theoretical  limit  is  established,  which  indicates  a  ca¬ 
pability  that  can  be  approached  only  asymptotically  and 
never  equalled  or  exceeded.  Second,  the  functional 
capability  curves  are  monitored  ifecause  without  a 
scientific  breakthrough,  these  could  not  be  exceeded  by 
any  specific  technique.  Third  and  of  highest  -signifi¬ 
cance,  the  design  functions  associated  with  the  specific 
techniques  under  investigation  are  examined  for  those 
components  and/or  subparameters  which  have  the  most 
influence  on  the  performance  of  the  equipment  After 
the  most  important  contributors  arc  identified,  fore¬ 
casts  involving  them  are  made  to  show  the  degree  of 
improvement  that  can  be  expected  in  the  basic  equip¬ 
ment  as  a  result  of  expected  improvement  in  the 
'component  or  design  limits. 

When  forecasts  are  made  at  the  subparameter  level, 
experimental  units,  laboratory  tests,  and  basic  research 
results  are  used  to  gain  an  insight  into  the  advances 
that  can  be  expected  in  the  technology.  An  example  of 
this  can  be  seen  in  several  of  the  gas-turbine  curves 
accompanying^  this  article. 

Finally,  technical  fields  related  or  contributing  to  the 
one  under  investigation  are  studied  to  find  out  if 
technological  advances  in  these  fields  may  be  ahead  of 
the  one  being  investigated.  An  instance  of  this  pro¬ 
cedure  is  shown  by  the  curves  of  aircraft  gas-turbine 
'technology  verr  marine.  This  procedure  bears  fruit 
only  in  relatively  few  cases,  such  as  when  compasisons 
are  made  with  advances  in  the  ajrcraft  and  space  fields, 
where  progress  is  the  result  of  spending  unusually  large 
amounts  of  money,  wei*  beyond  that  normally  expected 
under  less  urgent  conditions. 
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The  hamui  element  in  forecaeting  is  perhaps  the 
moat  difficuh  aspect  to  handle.  When  the  subject  of 
tedmolc^cal  !  forecasting  is  brought  up,  the  first  re- 
actimi  one  usually  gets  is  skepticism.  Surdy,  nobody 
claims  that  the  procedures  we  have  described  give  a 
dear  lode  into  the  future.  Nevertheless,  the  fact  that 
our  window  may  be  fogged  a  little  should  not  lead  one 
to  bdieve  th4t  we  have  no  window  at  ail.  And  the  lack 
of  certainty  with  reqwet  to  the  future  should  not  take 
away  from  the  value  of  technological  forecasting  for 
long>rahge  planning  purposes. 

There  is  little  doubt  that  if  we  can  ddineate  factors 
that  hsve  conti^ibuted  to  state-of-the-art  advancement, 
fnd  then  plot  their  progress  over  the  years  to  establish 
'  trends,  ihese  trends  will  continue  unless  deformed  by 
external  constraints  such  as  the  reaching  of  a  theor¬ 
etical  limit  or  the,  arrival  of  a  new  technique  of 
dearly  superior  capabilities. 

In  the  broadest  sense,  if  technological  forecasting  is 
looked  upon  as  movement  or  progress  rather  than  the 
upward  march  of  numbers,  then  the  quantitative  aspects 
of  theiforccastb  become  relatively  unimportant.  A  trend 
will  have  been  established,  and  errors,  even  as  much 
as  $0  percent,  will  have  little  effect  on  the  usefulness 
of  the  material. 
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Another  problem,  which  is  ejsentiaUy  a  human  one,  | 
is  to  assess  where  the  levels  of  effort  which  have  been 
applied  in  the  past  to  some  techniques  will  continue  in 
the  future.  The  knowledge  hdd  by  some  workers  in  a 
particular  field  might  indicate  to  than  that  a  certain 
technique  is  in  disfavor  with  policy  makers.  Therefore 
support  for  this  technique  is  apt  to  be  lessened,  thereby 
imposing  an  artificially  depressed  slope  in  the  curve  of 
progress.  However,  periods  of  favor  and  disfavor  are 
apt  to  cancel  each  other  out,  and  where  progress  is 
needed  and  is  possible,  progress  will  usually  occur. 

Nobody  disputes  that  all  subjectivity  should  be 
ferreted  out  in  gathering  historical  data  and  in  generat¬ 
ing  forecasts.  It  is  equally  true  that  this  is  a  difficult  I 

thing  to  ask.  We  do  not  argue  that  subjectivity  cannot  I 

lead  in  some  instances  to  good  forecasts.  However,  j 

such  forecasts  are  not  reproducible  in  the  scientific  j 

sense  and  therefore  are  difficult  to  generalize  or  apply  | 

broadly.  In  a  practical  sense,  the  Navy  attempts  to  | 

walk  the  middle  ground  between  subjectivity  and  j 

statistical  extrapolation,  and  is  guided  by  as  much  | 

common  sense  as  a  situation  wiU  permit.  The  future,  j 

after  all,  is  simply  a  logical  extension  of  the  past  and  | 

present,  and  all  we  have  to  do  is  find  the  curve  to  j 

extrapolate.  j 
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naval  aviator,  and  served  as  a  patrol-plane  pilot  in  the 
Pacific  area  until  1957.  From  1957  to  1959  he  attended 
the  U.S.  Naval  Postgraduate  School,  receiving  the 
degree  of  M.S.  in  meteorology  in  1959.  From  1959  to 
1962  Commander  Hamilton  served  as  a  meteorologist 
aboard  the  Carrier  USS  ESSEX  (CVA-9),  in  the 
Atlantic  and  the  Mediterranean.  In  1%2  he  was  as¬ 
signed  as  a  research  meteorologist  at  the  Navy  Weather 
Research  Facility,  Norfolk,  Va.  From  1963  to  1966 
he  studied  at  the  University  of  Stockholm,  Sweden, 
receiving  the  degree  Filosofie  Licentiat  in  1966.  Since 
1966  Commander  Hamilton  has  held  his  present 
assignment  as  assistant  officer  in  charge.  Fleet  Numeri¬ 
cal  Weather  Facility,  Monterey,  Calif. 

Makvin  J.  CrreoN 

Mr.  Cetron  is  currently  responsible  for  the  Navy’s 
Technological  Forecast  efforts  and  is  located  in  the 
Advanced  Concepts  Branch,  Headquarters  Naval 
Material  Conunand.  He  holds  a  B.S.  from  Pennsylvania 
State  University,  an  V.S.  from  Columbia  University 
and  is  currently  completi>ig  his  Doctoral  studies  in 
R.  &  D.  Management  at  American  University.  His 
technical  experience  includes  positions  as  Head,  Plan¬ 
ning  Branch,  Navy  Marine  Engineering  Laboratory; 


Engineering  mistant  to  the  Teclmical  Director,  Naval 
Applied  Scieq^  Laboratory;  and  Head  of  the  Man¬ 
agement  Planning  Review  Branch  of  the  Applied 
Science  Laboratory.  Mr.  Cetron  has  published  ex¬ 
tensively  in  the  fields  of  Planning,  Resour  »  Allocation, 
Technological  Forecasting,  and  Operations  Research. 
Hr  IS  a  member  of  the  Military  Operations  Research 
Society;  American  Institute  of  Indoatrial  Engineers; 
the  Interservice  Technological  Forecarting  Method¬ 
ology  Group;  and  the  Interagency  Group  for  Research 
on  Research. 

Donald  F.  Shfth 

Mr.  Smith  received  the  degree  of  B.S.  in  mechanical 
engineering  from  Tufts  University.  His  experience 
includes  16  years  with  private  industry,  including 
Generel  Electric  Co.,  American  Engineering  Co.,  North 
American  Rayon  Corp.,  Globe  Aircraft  Corp.,  and  the 
Johns-Manville  Corp.  His  work  included  powerplant 
design  and  construction,  aircraft  manufacturing, 
machine  design,  process  equipment  design,  and  field 
engineering.  He  served  2^/^  years  on  active  duty  with 
the  Navy  as  an  ordnance  office,  and  training  officer. 
Mr.  Smith  worked  for  14  years  at  the  Naval  Air  De¬ 
velopment  Center,  JohnsviUe,  Pa.,  starting  as  a  project 
engineer  in  the  Aircraft  Armament  Laboratory, 
later  as  Branch  Head  in  the  Anti-Submarine  Warfare 
Laboratory,  then  became  Project  Manager  for  the 
A-NEW  Syutem  of  airborne  ASW  dectronics.  He  a'so 
served  as  Program  Admii.istrator  for  the  ASW 
Laboratory,  and  has  been  with  the  Annapolis  Division, 
Naval  Ship  Research  and  Devolopinent  Center  for 
2Yi  years.  Currently,  Mr.  Smith  is  the  Head,  Technical 
Planning  Staff,  of  the  Annapolis  Division  of  the  Naval 
Ship  R.  &  D.  Center,  where  he  is  concerned  with  de¬ 
veloping  amall  craft  and  air  cushion  vdiide  R.  It  D. 
programs. 
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